[GitHub] [hadoop] ferhui commented on pull request #3538: HDFS-16266. Add remote port information to HDFS audit log

[GitBox]

ferhui commented on pull request #3538:
URL: https://github.com/apache/hadoop/pull/3538#issuecomment-946419243


   It's good that port is optional.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?focusedWorklogId=59&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-59
 ]

ASF GitHub Bot logged work on HADOOP-17965:
---

Author: ASF GitHub Bot
Created on: 19/Oct/21 03:30
Start Date: 19/Oct/21 03:30
Worklog Time Spent: 10m 
  Work Description: iwasakims commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-946336668


   The asflicense warnings and test failures are not related to the patch. I 
could not reproduce the test failures on my local.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 59)
Time Spent: 40m  (was: 0.5h)

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] iwasakims commented on pull request #3562: HADOOP-17965. Fix documentation build failure using JDK 7 on branch-2.10.

[GitBox]

iwasakims commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-946336668


   The asflicense warnings and test failures are not related to the patch. I 
could not reproduce the test failures on my local.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=666590&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666590
 ]

ASF GitHub Bot logged work on HADOOP-15327:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 23:11
Start Date: 18/Oct/21 23:11
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-946237823


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  17m 40s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 40s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  23m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   2m 37s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   2m  7s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 56s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 21s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 10s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   6m 37s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m  2s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 32s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | -1 :x: |  javac  |   2m 32s | 
[/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  
hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 350 
unchanged - 0 fixed = 351 total (was 350)  |
   | +1 :green_heart: |  compile  |   2m  4s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | -1 :x: |  javac  |   2m  4s | 
[/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  
hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 1 new 
+ 337 unchanged - 0 fixed = 338 total (was 337)  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/blanks-eol.txt)
 |  The patch has 24 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   0m 51s | 
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt)
 |  hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 83 
new + 134 unchanged - 10 fixed = 217 total (was 144)  |
   | +1 :green_heart: |  mvnsite  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  xml  |   0m  1s |  |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  javadoc  |   1m 46s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 32s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | -1 :x: |  spotbugs  |   4m  2s | 
[/new-spotbugs-hadoop-mapred

[GitHub] [hadoop] hadoop-yetus commented on pull request #3259: HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4

[GitBox]

hadoop-yetus commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-946237823


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  17m 40s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 4 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 40s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  23m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   2m 37s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   2m  7s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 56s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 21s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 10s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   6m 37s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   3m  2s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 32s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | -1 :x: |  javac  |   2m 32s | 
[/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  
hadoop-mapreduce-project_hadoop-mapreduce-client-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 350 
unchanged - 0 fixed = 351 total (was 350)  |
   | +1 :green_heart: |  compile  |   2m  4s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | -1 :x: |  javac  |   2m  4s | 
[/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  
hadoop-mapreduce-project_hadoop-mapreduce-client-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 1 new 
+ 337 unchanged - 0 fixed = 338 total (was 337)  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/blanks-eol.txt)
 |  The patch has 24 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   0m 51s | 
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt)
 |  hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 83 
new + 134 unchanged - 10 fixed = 217 total (was 144)  |
   | +1 :green_heart: |  mvnsite  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  xml  |   0m  1s |  |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  javadoc  |   1m 46s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 32s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | -1 :x: |  spotbugs  |   4m  2s | 
[/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3259/3/artifact/out/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client.html)
 |  hadoop-mapreduce-project/hadoop-mapreduce-client generated 1 new + 0 
unchanged - 0 fixed = 1 total (was 0)  |
   | -1 :x: |  spotbugs  |   1m 22s | 
[/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-co

[GitHub] [hadoop] hadoop-yetus commented on pull request #3563: HDFS-16278. Make HDFS snapshot tools cross platform

[GitBox]

hadoop-yetus commented on pull request #3563:
URL: https://github.com/apache/hadoop/pull/3563#issuecomment-946215556


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  43m 53s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 9 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  36m 59s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   2m 52s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 25s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  61m 13s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 14s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 51s |  |  the patch passed  |
   | +1 :green_heart: |  cc  |   2m 51s |  |  the patch passed  |
   | +1 :green_heart: |  golang  |   2m 51s |  |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 51s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  mvnsite  |   0m 16s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 25s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  95m  2s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3563/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt)
 |  hadoop-hdfs-native-client in the patch failed.  |
   | +1 :green_heart: |  asflicense  |   0m 30s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 226m 47s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed CTEST tests | hdfs_tool_tests |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3563/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3563 |
   | Optional Tests | dupname asflicense compile cc mvnsite javac unit 
codespell golang |
   | uname | Linux bf40bbc856ef 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 646731f7eaaa18f4088c90c5f8eebbf30650dca9 |
   | Default Java | Red Hat, Inc.-1.8.0_302-b08 |
   | CTEST | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3563/1/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3563/1/testReport/ |
   | Max. process+thread count | 571 (vs. ulimit of 5500) |
   | modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U: 
hadoop-hdfs-project/hadoop-hdfs-native-client |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3563/1/console |
   | versions | git=2.9.5 maven=3.6.3 |
   | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #3560: YARN-10958. Use correct configuration for Group service init in CSMappingPlacementRule

[GitBox]

hadoop-yetus commented on pull request #3560:
URL: https://github.com/apache/hadoop/pull/3560#issuecomment-946165686


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 56s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  13m 26s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  27m 30s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  29m 23s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  24m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   4m 30s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   3m 13s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   2m 18s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 48s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   5m 30s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  25m 49s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 18s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  28m 43s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  28m 43s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  24m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  24m 33s |  |  the patch passed  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3560/2/artifact/out/blanks-eol.txt)
 |  The patch has 6 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   4m 23s | 
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3560/2/artifact/out/results-checkstyle-root.txt)
 |  root: The patch generated 6 new + 12 unchanged - 0 fixed = 18 total (was 
12)  |
   | +1 :green_heart: |  mvnsite  |   3m  8s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   2m 14s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 46s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   5m 56s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  26m 18s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  19m 48s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  | 110m  0s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 56s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 372m 27s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3560/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3560 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 14369548ee43 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 9674331ae2bb6b5c8ba649514875d4f569de5f1a |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3560/2/testReport/ |
   | Max. process+thread count | 3003 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-se

[GitHub] [hadoop] hadoop-yetus commented on pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

hadoop-yetus commented on pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#issuecomment-946165029


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  17m  5s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  34m  6s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  5s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 53s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 47s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m  0s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 45s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   2m 29s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  28m 29s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 12s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 27s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   1m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 11s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   1m 11s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 58s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   1m 21s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 59s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   3m  7s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  32m 30s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 152m 44s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/7/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 43s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 282m 49s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.yarn.server.resourcemanager.TestRMAdminService 
|
   |   | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesContainers |
   |   | 
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits |
   |   | hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/7/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3500 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 2030aabc72a3 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 522086313c2fb96dd16e2f09dc82d51f6fec236f |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/7/testReport/ |
   | Max. process+thread count | 937 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-serv

[GitHub] [hadoop] hadoop-yetus commented on pull request #3550: YARN-10907. Minimize usages of AbstractCSQueue#csContext

[GitBox]

hadoop-yetus commented on pull request #3550:
URL: https://github.com/apache/hadoop/pull/3550#issuecomment-946096452


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 51s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 21 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  |  Maven dependency ordering for branch  |
   | -1 :x: |  mvninstall  |   0m 25s | 
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-mvninstall-root.txt)
 |  root in trunk failed.  |
   | -1 :x: |  compile  |   0m 24s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  hadoop-yarn in trunk failed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.  |
   | -1 :x: |  compile  |   0m 25s | 
[/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  hadoop-yarn in trunk failed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.  |
   | -0 :warning: |  checkstyle  |   0m 23s | 
[/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
 |  The patch fails to run checkstyle in hadoop-yarn  |
   | -1 :x: |  mvnsite  |   0m 25s | 
[/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt)
 |  hadoop-yarn-api in trunk failed.  |
   | -1 :x: |  mvnsite  |   0m 24s | 
[/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed.  |
   | -1 :x: |  javadoc  |   0m 24s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  hadoop-yarn-api in trunk failed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.  |
   | -1 :x: |  javadoc  |   0m 25s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.  |
   | -1 :x: |  javadoc  |   0m 25s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  hadoop-yarn-api in trunk failed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.  |
   | -1 :x: |  javadoc  |   0m 24s | 
[/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3550/3/artifact/out/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  hadoop-yarn-server-resourcemanager in trunk failed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.  |
   | -1 :x: |  spotbugs  |   0m 24s | 
[/branch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt](h

[jira] [Work logged] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?focusedWorklogId=666486&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666486
 ]

ASF GitHub Bot logged work on HADOOP-17965:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 19:19
Start Date: 18/Oct/21 19:19
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-946087975


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 12s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
   ||| _ branch-2.10 Compile Tests _ |
   | +0 :ok: |  mvndep  |   2m 21s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  12m 31s |  branch-2.10 passed  |
   | +1 :green_heart: |  compile  |  13m  2s |  branch-2.10 passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  10m 42s |  branch-2.10 passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   | +1 :green_heart: |  mvnsite  |   5m 42s |  branch-2.10 passed  |
   | +1 :green_heart: |  javadoc  |   3m 38s |  branch-2.10 passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   2m 54s |  branch-2.10 passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |  14m 59s |  the patch passed with JDK Azul 
Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  14m 59s |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 27s |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   | +1 :green_heart: |  javac  |  11m 27s |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   5m 37s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  3s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  javadoc  |   3m 43s |  the patch passed with JDK Azul 
Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 106m 13s |  hadoop-yarn-project in the patch failed.  |
   | -1 :x: |  unit  | 139m 37s |  hadoop-mapreduce-project in the patch 
failed.  |
   | -1 :x: |  asflicense  |   0m 58s |  The patch generated 1 ASF License 
warnings.  |
   |  |   | 347m 16s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3562 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient xml |
   | uname | Linux b3bb2d7bf561 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10 / b7bb43f |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 |
   | unit | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-unit-hadoop-yarn-project.txt
 |
   | unit | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-unit-hadoop-mapreduce-project.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/testReport/ |
   | asflicense | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-asflicense-problems.txt
 |
   | Max. process+thread count | 1222 (vs. ulimit of 5500) |
   | modules | C: hadoop-yarn-project hadoop-mapreduce-project U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/console |
   | versions | git=2.7.4 maven=3.3.9 |
   | Powered by | Apache Yetu

[GitHub] [hadoop] hadoop-yetus commented on pull request #3562: HADOOP-17965. Fix documentation build failure using JDK 7 on branch-2.10.

[GitBox]

hadoop-yetus commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-946087975


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |::|--:|:|:|
   | +0 :ok: |  reexec  |   1m 12s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files 
found.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any 
@author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
   ||| _ branch-2.10 Compile Tests _ |
   | +0 :ok: |  mvndep  |   2m 21s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  12m 31s |  branch-2.10 passed  |
   | +1 :green_heart: |  compile  |  13m  2s |  branch-2.10 passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  compile  |  10m 42s |  branch-2.10 passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   | +1 :green_heart: |  mvnsite  |   5m 42s |  branch-2.10 passed  |
   | +1 :green_heart: |  javadoc  |   3m 38s |  branch-2.10 passed with JDK 
Azul Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   2m 54s |  branch-2.10 passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 22s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 33s |  the patch passed  |
   | +1 :green_heart: |  compile  |  14m 59s |  the patch passed with JDK Azul 
Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javac  |  14m 59s |  the patch passed  |
   | +1 :green_heart: |  compile  |  11m 27s |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   | +1 :green_heart: |  javac  |  11m 27s |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   5m 37s |  the patch passed  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace 
issues.  |
   | +1 :green_heart: |  xml  |   0m  3s |  The patch has no ill-formed XML 
file.  |
   | +1 :green_heart: |  javadoc  |   3m 43s |  the patch passed with JDK Azul 
Systems, Inc.-1.7.0_262-b10  |
   | +1 :green_heart: |  javadoc  |   2m 44s |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10  |
   ||| _ Other Tests _ |
   | -1 :x: |  unit  | 106m 13s |  hadoop-yarn-project in the patch failed.  |
   | -1 :x: |  unit  | 139m 37s |  hadoop-mapreduce-project in the patch 
failed.  |
   | -1 :x: |  asflicense  |   0m 58s |  The patch generated 1 ASF License 
warnings.  |
   |  |   | 347m 16s |   |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3562 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient xml |
   | uname | Linux b3bb2d7bf561 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | branch-2.10 / b7bb43f |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 |
   | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, 
Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 |
   | unit | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-unit-hadoop-yarn-project.txt
 |
   | unit | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-unit-hadoop-mapreduce-project.txt
 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/testReport/ |
   | asflicense | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/artifact/out/patch-asflicense-problems.txt
 |
   | Max. process+thread count | 1222 (vs. ulimit of 5500) |
   | modules | C: hadoop-yarn-project hadoop-mapreduce-project U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3562/1/console |
   | versions | git=2.7.4 maven=3.3.9 |
   | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



--

[GitHub] [hadoop] GauthamBanasandra opened a new pull request #3563: HDFS-16278. Make HDFS snapshot tools cross platform

[GitBox]

GauthamBanasandra opened a new pull request #3563:
URL: https://github.com/apache/hadoop/pull/3563


   
   
   ### Description of PR
   The source files for `hdfs_createSnapshot`, `hdfs_disallowSnapshot` and 
`hdfs_renameSnapshot` uses getopt for parsing the command line arguments. 
getopt is available only on Linux and thus, isn't cross platform. We need to 
replace getopt with boost::program_options to make these tools cross platform.
   
   ### How was this patch tested?
   In progress.
   
   ### For code changes:
   
   - [x] Does the title or this PR starts with the corresponding JIRA issue id 
(e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the 
endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, 
`NOTICE-binary` files?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17880) Build 2.10.x with docker

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17880?focusedWorklogId=666425&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666425
 ]

ASF GitHub Bot logged work on HADOOP-17880:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 17:17
Start Date: 18/Oct/21 17:17
Worklog Time Spent: 10m 
  Work Description: GauthamBanasandra commented on a change in pull request 
#3535:
URL: https://github.com/apache/hadoop/pull/3535#discussion_r731149203



##
File path: dev-support/docker/pkg-resolver/resolve.py
##
@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Platform package dependency resolver for building Apache Hadoop.
+"""
+
+from __future__ import print_function

Review comment:
   The cause for the pylint warning was that we use Python 3 in Hadoop 3.x 
and we're still using Python 2 in Hadoop 2.x. Instead of importing the 
`print_function`, could you please rewrite the print statement using Python 2 
syntax? I feel this is a better approach since all the Python scripts in Hadoop 
2.x is written in Python 2.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666425)
Time Spent: 10.5h  (was: 10h 20m)

> Build 2.10.x with docker
> 
>
> Key: HADOOP-17880
> URL: https://issues.apache.org/jira/browse/HADOOP-17880
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.10.0
> Environment: mac os x86_64
>Reporter: baizhendong
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>
> 1. currently, we build the hadoop 2.10.0 with docker machine, and must 
> install Virtual Box, and for hadoop 3.x, just build with docker only.
> 2. besides this, the docker image dependency is out of date, and some of them 
> is unavaialble, for example – jdk7
> 3. but just building hadoop 2.10.0 with hadoop 3.x build script without 
> modification is not working, for the protocol buffer version is not 2.5.0, 
> and it's not work for native build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] GauthamBanasandra commented on a change in pull request #3535: HADOOP-17880. Build 2.10.x with docker

[GitBox]

GauthamBanasandra commented on a change in pull request #3535:
URL: https://github.com/apache/hadoop/pull/3535#discussion_r731149203



##
File path: dev-support/docker/pkg-resolver/resolve.py
##
@@ -0,0 +1,62 @@
+#!/usr/bin/env python3
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Platform package dependency resolver for building Apache Hadoop.
+"""
+
+from __future__ import print_function

Review comment:
   The cause for the pylint warning was that we use Python 3 in Hadoop 3.x 
and we're still using Python 2 in Hadoop 2.x. Instead of importing the 
`print_function`, could you please rewrite the print statement using Python 2 
syntax? I feel this is a better approach since all the Python scripts in Hadoop 
2.x is written in Python 2.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

hadoop-yetus commented on pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#issuecomment-945971256


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 50s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  31m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  3s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 54s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 52s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 49s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m  5s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | -1 :x: |  mvninstall  |   0m 29s | 
[/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed.  |
   | -1 :x: |  compile  |   0m 32s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.  |
   | -1 :x: |  javac  |   0m 32s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.  |
   | -1 :x: |  compile  |   0m 29s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.  |
   | -1 :x: |  javac  |   0m 29s | 
[/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
 |  hadoop-yarn-server-resourcemanager in the patch failed with JDK Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.  |
   | -1 :x: |  blanks  |   0m  0s | 
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/blanks-eol.txt)
 |  The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix 
<>. Refer https://git-scm.com/docs/git-apply  |
   | -0 :warning: |  checkstyle  |   0m 41s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3551/6/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-serv

[jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=666391&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666391
 ]

ASF GitHub Bot logged work on HADOOP-15327:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 15:59
Start Date: 18/Oct/21 15:59
Worklog Time Spent: 10m 
  Work Description: szilard-nemeth commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-945923248


   Hi @jasonwzs,
   Do you have any idea how to fix the shading maven issue? 
   I'm kind of far from a maven expert :S 
   You can wait for the latest build result (just rebased the branch to trunk 
and pushed a commit) or you can also check the previous result if it's still 
accessible.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666391)
Time Spent: 6h  (was: 5h 50m)

> Upgrade MR ShuffleHandler to use Netty4
> ---
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Szilard Nemeth
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, 
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch, 
> HADOOP-15327.005.patch, 
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log, 
> testfailure-testMapFileAccess-emptyresponse.zip, 
> testfailure-testReduceFromPartialMem.zip
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[Szilard Nemeth (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430075#comment-17430075
 ] 

Szilard Nemeth edited comment on HADOOP-15327 at 10/18/21, 3:59 PM:


Hi [~zhenshan.wen] ,

I'm planning to fix the maven shading issue in the coming weeks, as soon as 
possible.

Also, I'd appreciate if you could help me to find out what to fix to get rid of 
the Maven shading issue. 

Same thing I asked for here: 
[https://github.com/apache/hadoop/pull/3259#issuecomment-945923248]


was (Author: snemeth):
Hi [~zhenshan.wen] ,

I'm planning to fix the maven shading issue in the coming weeks, as soon as 
possible.

Also, I'd appreciate if you could help me to find out what to fix to get rid of 
the Maven shading issue. 

Same thing I asked here: 
https://github.com/apache/hadoop/pull/3259#issuecomment-945923248

> Upgrade MR ShuffleHandler to use Netty4
> ---
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Szilard Nemeth
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, 
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch, 
> HADOOP-15327.005.patch, 
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log, 
> testfailure-testMapFileAccess-emptyresponse.zip, 
> testfailure-testReduceFromPartialMem.zip
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[Szilard Nemeth (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430075#comment-17430075
 ] 

Szilard Nemeth edited comment on HADOOP-15327 at 10/18/21, 3:59 PM:


Hi [~zhenshan.wen] ,

I'm planning to fix the maven shading issue in the coming weeks, as soon as 
possible.

Also, I'd appreciate if you could help me to find out what to fix to get rid of 
the Maven shading issue. 

Same thing I asked here: 
https://github.com/apache/hadoop/pull/3259#issuecomment-945923248


was (Author: snemeth):
Hi [~zhenshan.wen] ,

I'm planning to fix the maven shading issue in the coming weeks, as soon as 
possible.

> Upgrade MR ShuffleHandler to use Netty4
> ---
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Szilard Nemeth
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, 
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch, 
> HADOOP-15327.005.patch, 
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log, 
> testfailure-testMapFileAccess-emptyresponse.zip, 
> testfailure-testReduceFromPartialMem.zip
>
>  Time Spent: 6h
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on pull request #3259: HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4

[GitBox]

szilard-nemeth commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-945923248


   Hi @jasonwzs,
   Do you have any idea how to fix the shading maven issue? 
   I'm kind of far from a maven expert :S 
   You can wait for the latest build result (just rebased the branch to trunk 
and pushed a commit) or you can also check the previous result if it's still 
accessible.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15327?focusedWorklogId=666390&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666390
 ]

ASF GitHub Bot logged work on HADOOP-15327:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 15:57
Start Date: 18/Oct/21 15:57
Worklog Time Spent: 10m 
  Work Description: szilard-nemeth commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-945921704


   > Thanks @szilard-nemeth . Will there be any issue if we backport this 
change to 3.2.x?
   
   Hopefully not but we need to take care of this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666390)
Time Spent: 5h 50m  (was: 5h 40m)

> Upgrade MR ShuffleHandler to use Netty4
> ---
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Szilard Nemeth
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, 
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch, 
> HADOOP-15327.005.patch, 
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log, 
> testfailure-testMapFileAccess-emptyresponse.zip, 
> testfailure-testReduceFromPartialMem.zip
>
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on pull request #3259: HADOOP-15327. Upgrade MR ShuffleHandler to use Netty4

[GitBox]

szilard-nemeth commented on pull request #3259:
URL: https://github.com/apache/hadoop/pull/3259#issuecomment-945921704


   > Thanks @szilard-nemeth . Will there be any issue if we backport this 
change to 3.2.x?
   
   Hopefully not but we need to take care of this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15327) Upgrade MR ShuffleHandler to use Netty4

[Szilard Nemeth (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430075#comment-17430075
 ] 

Szilard Nemeth commented on HADOOP-15327:
-

Hi [~zhenshan.wen] ,

I'm planning to fix the maven shading issue in the coming weeks, as soon as 
possible.

> Upgrade MR ShuffleHandler to use Netty4
> ---
>
> Key: HADOOP-15327
> URL: https://issues.apache.org/jira/browse/HADOOP-15327
> Project: Hadoop Common
>  Issue Type: Sub-task
>Reporter: Xiaoyu Yao
>Assignee: Szilard Nemeth
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-15327.001.patch, HADOOP-15327.002.patch, 
> HADOOP-15327.003.patch, HADOOP-15327.004.patch, HADOOP-15327.005.patch, 
> HADOOP-15327.005.patch, 
> getMapOutputInfo_BlockingOperationException_awaitUninterruptibly.log, 
> testfailure-testMapFileAccess-emptyresponse.zip, 
> testfailure-testReduceFromPartialMem.zip
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> This way, we can remove the dependencies on the netty3 (jboss.netty)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731065132



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/BasicQueueProperties.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+public class BasicQueueProperties {

Review comment:
   OK, fixed this as well.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

hadoop-yetus commented on pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#issuecomment-945902428


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 52s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  32m 28s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  3s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 56s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 48s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m  0s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 47s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 41s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   2m  1s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m  5s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m  0s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 59s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   0m 59s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 48s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   0m 48s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 38s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 52s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 37s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 33s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 59s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m 25s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  95m 12s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 32s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 184m 39s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/6/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3500 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 9abb9485183f 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 4b46cc7006843ff84df06bfed4f3421c95c49c71 |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/6/testReport/ |
   | Max. process+thread count | 960 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/6/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731055386



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/BasicQueueProperties.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+public class BasicQueueProperties {

Review comment:
   This actually makes sense. Let me check what I can do to fix this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17969) NFS unable to bind to chosen address

[Adam Soroka (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Soroka updated HADOOP-17969:
-
Affects Version/s: 3.3.1

> NFS unable to bind to chosen address
> 
>
> Key: HADOOP-17969
> URL: https://issues.apache.org/jira/browse/HADOOP-17969
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: nfs
>Affects Versions: 3.1.1, 3.3.1
>Reporter: Adam Soroka
>Priority: Minor
>  Labels: nfsgateway
>
> [https://lists.apache.org/list.html?u...@hadoop.apache.org:2021-10]
>  
> NFS Gateway appears not have a configuration setting for selecting which 
> IP(s) to which to bind on a multihomed machine. I would like to offer a patch 
> to add this feature in line with other similar settings for HDFS components 
> that can bind to a chosen address. Or even better, I would _love_ to be told 
> I am wrong and that this config setting _does_ exist. :) I will attempt to 
> create a patch, but I have not contributed to Hadoop before, so I will need 
> handholding.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731046875



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java
##
@@ -265,31 +242,31 @@ public CSQueueMetrics getMetrics() {
 
   @Override
   public String getQueueShortName() {
-return queueName;
+return basicQueueProperties.getQueueName();
   }
 
   @Override
   public String getQueueName() {
-return queueName;
+return basicQueueProperties.getQueueName();
   }
 
   @Override
-  public PrivilegedEntity getPrivilegedEntity() {
-return queueEntity;
+  public CSQueue getParent() {

Review comment:
   Which comment are you referring to? Can you link it please?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731046196



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueAllocationSettings.java
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.hadoop.yarn.api.records.Resource;
+import org.apache.hadoop.yarn.api.records.ResourceInformation;
+import org.apache.hadoop.yarn.util.resource.ResourceUtils;
+import org.apache.hadoop.yarn.util.resource.Resources;
+
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.UNDEFINED;
+
+public class QueueAllocationSettings {
+  private final Resource minimumAllocation;
+  private volatile Resource maximumAllocation;
+
+  public QueueAllocationSettings(CapacitySchedulerContext csContext) {
+this.minimumAllocation = csContext.getMinimumResourceCapability();
+  }
+
+  void setupMaximumAllocation(CapacitySchedulerConfiguration csConf, String 
queuePath,
+  CSQueue parent, CapacitySchedulerContext csContext) {
+/* YARN-10869: When using AutoCreatedLeafQueues, the passed configuration

Review comment:
   Well, I guess we can remove that comment if YARN-10929 gets merged. We 
can also include this in the follow-up jira. Is that okay?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Created] (HADOOP-17969) NFS unable to bind to chosen address

[Adam Soroka (Jira)]

Adam Soroka created HADOOP-17969:


 Summary: NFS unable to bind to chosen address
 Key: HADOOP-17969
 URL: https://issues.apache.org/jira/browse/HADOOP-17969
 Project: Hadoop Common
  Issue Type: Improvement
  Components: nfs
Affects Versions: 3.1.1
Reporter: Adam Soroka


[https://lists.apache.org/list.html?u...@hadoop.apache.org:2021-10]

 

NFS Gateway appears not have a configuration setting for selecting which IP(s) 
to which to bind on a multihomed machine. I would like to offer a patch to add 
this feature in line with other similar settings for HDFS components that can 
bind to a chosen address. Or even better, I would _love_ to be told I am wrong 
and that this config setting _does_ exist. :) I will attempt to create a patch, 
but I have not contributed to Hadoop before, so I will need handholding.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731044314



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueAppLifetimeAndLimitSettings.java
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.hadoop.yarn.exceptions.YarnRuntimeException;
+
+public class QueueAppLifetimeAndLimitSettings {
+  // -1 indicates lifetime is disabled
+  private volatile long maxApplicationLifetime = -1;
+  private volatile long defaultApplicationLifetime = -1;
+
+  // Indicates if this queue's default lifetime was set by a config property,
+  // either at this level or anywhere in the queue's hierarchy.
+  private volatile boolean defaultAppLifetimeWasSpecifiedInConfig = false;
+
+  private int maxParallelApps;
+
+  public QueueAppLifetimeAndLimitSettings(CapacitySchedulerConfiguration 
configuration,
+  AbstractCSQueue q, String queuePath) {
+// Store max parallel apps property
+this.maxParallelApps = configuration.getMaxParallelAppsForQueue(queuePath);
+this.maxApplicationLifetime = getInheritedMaxAppLifetime(q, configuration);
+this.defaultApplicationLifetime = getInheritedDefaultAppLifetime(q, 
queuePath, configuration, 
+maxApplicationLifetime);
+  }
+
+  private long getInheritedMaxAppLifetime(CSQueue q, 
CapacitySchedulerConfiguration conf) {
+CSQueue parentQ = q.getParent();
+long maxAppLifetime = conf.getMaximumLifetimePerQueue(q.getQueuePath());
+
+// If q is the root queue, then get max app lifetime from conf.
+if (parentQ == null) {
+  return maxAppLifetime;
+}
+
+// If this is not the root queue, get this queue's max app lifetime
+// from the conf. The parent's max app lifetime will be used if it's
+// not set for this queue.
+// A value of 0 will override the parent's value and means no max lifetime.
+// A negative value means that the parent's max should be used.
+long parentsMaxAppLifetime = parentQ.getMaximumApplicationLifetime();
+return (maxAppLifetime >= 0) ? maxAppLifetime : parentsMaxAppLifetime;
+  }
+
+  private long getInheritedDefaultAppLifetime(CSQueue q,
+  String queuePath, CapacitySchedulerConfiguration conf, long 
myMaxAppLifetime) {
+CSQueue parentQ = q.getParent();
+long defaultAppLifetime = conf.getDefaultLifetimePerQueue(queuePath);
+defaultAppLifetimeWasSpecifiedInConfig =

Review comment:
   This falls to the same category I described above, I didn't want to 
include any refactor changes in this PR. Are you fine with a follow-up jira? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731042502



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueAllocationSettings.java
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.hadoop.yarn.api.records.Resource;
+import org.apache.hadoop.yarn.api.records.ResourceInformation;
+import org.apache.hadoop.yarn.util.resource.ResourceUtils;
+import org.apache.hadoop.yarn.util.resource.Resources;
+
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.UNDEFINED;
+
+public class QueueAllocationSettings {

Review comment:
   added javadoc




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731039920



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueAppLifetimeAndLimitSettings.java
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.hadoop.yarn.exceptions.YarnRuntimeException;
+
+public class QueueAppLifetimeAndLimitSettings {

Review comment:
   added javadoc




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731037561



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueNodeLabelsSettings.java
##
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.util.Sets;
+import 
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager;
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.ROOT;
+import java.io.IOException;
+import java.util.Set;
+
+public class QueueNodeLabelsSettings {

Review comment:
   fixed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731035100



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/BasicQueueProperties.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+public class BasicQueueProperties {
+  private final String queueName;
+  private final String queuePath;

Review comment:
   See my comment above :)

##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/BasicQueueProperties.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+public class BasicQueueProperties {
+  private final String queueName;
+  private final String queuePath;
+  private CSQueue parent;
+
+  public BasicQueueProperties(CSQueue parent, String queueName) {
+this.parent = parent;
+this.queueName = queueName;
+this.queuePath = ((parent == null) ? "" : (parent.getQueuePath() + ".")) + 
this.queueName;

Review comment:
   See my comment above :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731034595



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueNodeLabelsSettings.java
##
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.util.Sets;
+import 
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager;
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.ROOT;
+import java.io.IOException;
+import java.util.Set;
+
+public class QueueNodeLabelsSettings {
+  private final CSQueue parent;
+  private final String queuePath;
+  private final CapacitySchedulerContext csContext;
+  private Set accessibleLabels;
+  private Set configuredNodeLabels;
+  private String defaultLabelExpression;
+
+  public QueueNodeLabelsSettings(CapacitySchedulerConfiguration configuration,
+  CSQueue parent,
+  String queuePath,
+  CapacitySchedulerContext csContext) throws IOException {
+this.parent = parent;
+this.queuePath = queuePath;
+this.csContext = csContext;
+initializeNodeLabels(configuration);
+  }
+
+  private void initializeNodeLabels(CapacitySchedulerConfiguration 
configuration)
+  throws IOException {
+initializeAccessibleLabels(configuration);
+initializeDefaultLabelExpression(configuration);
+initializeConfiguredNodeLabels();
+validateNodeLabels();
+  }
+
+  private void initializeAccessibleLabels(CapacitySchedulerConfiguration 
configuration) {
+this.accessibleLabels = configuration.getAccessibleNodeLabels(queuePath);
+// Inherit labels from parent if not set
+if (this.accessibleLabels == null && parent != null) {
+  this.accessibleLabels = parent.getAccessibleNodeLabels();
+}
+  }
+
+  private void initializeDefaultLabelExpression(CapacitySchedulerConfiguration 
configuration) {
+this.defaultLabelExpression = 
configuration.getDefaultNodeLabelExpression(queuePath);
+// If the accessible labels is not null and the queue has a parent with a
+// similar set of labels copy the defaultNodeLabelExpression from the 
parent
+if (this.accessibleLabels != null && parent != null
+&& this.defaultLabelExpression == null &&
+this.accessibleLabels.containsAll(parent.getAccessibleNodeLabels())) {
+  this.defaultLabelExpression = parent.getDefaultNodeLabelExpression();
+}
+  }
+
+  private void initializeConfiguredNodeLabels() {
+if (csContext.getCapacitySchedulerQueueManager() != null
+&& 
csContext.getCapacitySchedulerQueueManager().getConfiguredNodeLabels() != null) 
{
+  if (queuePath.equals(ROOT)) {
+this.configuredNodeLabels = 
csContext.getCapacitySchedulerQueueManager()
+.getConfiguredNodeLabels().getAllConfiguredLabels();
+  } else {
+this.configuredNodeLabels = 
csContext.getCapacitySchedulerQueueManager()
+.getConfiguredNodeLabels().getLabelsByQueue(queuePath);
+  }
+} else {
+  // Fallback to suboptimal but correct logic
+  this.configuredNodeLabels = 
csContext.getConfiguration().getConfiguredNodeLabels(queuePath);
+}
+  }
+
+  private void validateNodeLabels() throws IOException {
+// Check if labels of this queue is a subset of parent queue, only do this
+// when the queue in question is not root
+if (isNotRoot()) {
+  if (parent.getAccessibleNodeLabels() != null && !parent
+  .getAccessibleNodeLabels().contains(RMNodeLabelsManager.ANY)) {
+// If parent isn't "*", child shouldn't be "*" too
+if (this.getAccessibleNodeLabels().contains(RMNodeLabelsManager.ANY)) {
+  throw new IOException("Parent's accessible queue is not ANY(*), "
+  + "but child's accessible queue is " + RMNodeLabelsManager.ANY);
+} else {
+  Set diff = Sets.difference(this.getAccessibleNodeLabels(),
+ 

[GitHub] [hadoop] hadoop-yetus commented on pull request #3430: YARN-10942. Move AbstractCSQueue fields to separate objects that are tracking usage

[GitBox]

hadoop-yetus commented on pull request #3430:
URL: https://github.com/apache/hadoop/pull/3430#issuecomment-945872086


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 48s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  31m 54s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  4s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 55s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 48s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m  1s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 49s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 42s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 51s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m  1s |  |  branch has no errors 
when building and testing our client artifacts.  |
   | -0 :warning: |  patch  |  20m 23s |  |  Used diff version of patch file. 
Binary files and potentially other changes not applied. Please rebase and 
squash commits if necessary.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 51s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 54s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   0m 54s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 47s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   0m 47s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | -0 :warning: |  checkstyle  |   0m 40s | 
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3430/8/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
 The patch generated 3 new + 67 unchanged - 4 fixed = 70 total (was 71)  |
   | +1 :green_heart: |  mvnsite  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 38s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 34s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 52s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  19m 52s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  95m 34s |  |  
hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 33s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 182m 44s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3430/8/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3430 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 10b14f7eae3b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / db66cc64dbdbd29fc7d14736dc4e6b51bc5717a4 |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3430/8/testReport/ |
   | Max. process+thread count | 965 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
ha

[GitHub] [hadoop] szilard-nemeth commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

szilard-nemeth commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r731015567



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueNodeLabelsSettings.java
##
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.util.Sets;
+import 
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager;
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.ROOT;
+import java.io.IOException;
+import java.util.Set;
+
+public class QueueNodeLabelsSettings {
+  private final CSQueue parent;
+  private final String queuePath;
+  private final CapacitySchedulerContext csContext;
+  private Set accessibleLabels;
+  private Set configuredNodeLabels;
+  private String defaultLabelExpression;
+
+  public QueueNodeLabelsSettings(CapacitySchedulerConfiguration configuration,
+  CSQueue parent,
+  String queuePath,
+  CapacitySchedulerContext csContext) throws IOException {
+this.parent = parent;
+this.queuePath = queuePath;
+this.csContext = csContext;
+initializeNodeLabels(configuration);
+  }
+
+  private void initializeNodeLabels(CapacitySchedulerConfiguration 
configuration)
+  throws IOException {
+initializeAccessibleLabels(configuration);
+initializeDefaultLabelExpression(configuration);
+initializeConfiguredNodeLabels();
+validateNodeLabels();
+  }
+
+  private void initializeAccessibleLabels(CapacitySchedulerConfiguration 
configuration) {
+this.accessibleLabels = configuration.getAccessibleNodeLabels(queuePath);
+// Inherit labels from parent if not set
+if (this.accessibleLabels == null && parent != null) {
+  this.accessibleLabels = parent.getAccessibleNodeLabels();
+}
+  }
+
+  private void initializeDefaultLabelExpression(CapacitySchedulerConfiguration 
configuration) {
+this.defaultLabelExpression = 
configuration.getDefaultNodeLabelExpression(queuePath);
+// If the accessible labels is not null and the queue has a parent with a
+// similar set of labels copy the defaultNodeLabelExpression from the 
parent
+if (this.accessibleLabels != null && parent != null
+&& this.defaultLabelExpression == null &&
+this.accessibleLabels.containsAll(parent.getAccessibleNodeLabels())) {
+  this.defaultLabelExpression = parent.getDefaultNodeLabelExpression();
+}
+  }
+
+  private void initializeConfiguredNodeLabels() {
+if (csContext.getCapacitySchedulerQueueManager() != null
+&& 
csContext.getCapacitySchedulerQueueManager().getConfiguredNodeLabels() != null) 
{
+  if (queuePath.equals(ROOT)) {
+this.configuredNodeLabels = 
csContext.getCapacitySchedulerQueueManager()
+.getConfiguredNodeLabels().getAllConfiguredLabels();
+  } else {
+this.configuredNodeLabels = 
csContext.getCapacitySchedulerQueueManager()
+.getConfiguredNodeLabels().getLabelsByQueue(queuePath);
+  }
+} else {
+  // Fallback to suboptimal but correct logic
+  this.configuredNodeLabels = 
csContext.getConfiguration().getConfiguredNodeLabels(queuePath);
+}
+  }
+
+  private void validateNodeLabels() throws IOException {
+// Check if labels of this queue is a subset of parent queue, only do this
+// when the queue in question is not root
+if (isNotRoot()) {
+  if (parent.getAccessibleNodeLabels() != null && !parent
+  .getAccessibleNodeLabels().contains(RMNodeLabelsManager.ANY)) {
+// If parent isn't "*", child shouldn't be "*" too
+if (this.getAccessibleNodeLabels().contains(RMNodeLabelsManager.ANY)) {
+  throw new IOException("Parent's accessible queue is not ANY(*), "
+  + "but child's accessible queue is " + RMNodeLabelsManager.ANY);
+} else {
+  Set diff = Sets.difference(this.getAccessibleNodeLabels(),
+ 

[jira] [Work logged] (HADOOP-17953) S3A: ITestS3AFileContextStatistics test to lookup global or per-bucket configuration for encryption algorithm

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17953?focusedWorklogId=666331&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666331
 ]

ASF GitHub Bot logged work on HADOOP-17953:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 14:34
Start Date: 18/Oct/21 14:34
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3525:
URL: https://github.com/apache/hadoop/pull/3525#issuecomment-945840872


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 39s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 3 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 44s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  20m 39s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  21m 13s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  18m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   3m 38s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   2m 35s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 50s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   3m 53s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 35s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m  0s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  22m  0s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  19m 26s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  19m 26s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   3m 39s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   2m 23s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 36s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 17s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   4m  3s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m  3s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  17m 32s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   2m 35s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m  0s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 210m 47s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3525/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3525 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux e01a112bdfc6 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 9f2d123c646139feae246076bb61bb91c15dfa9b |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3525/2/testReport/ |
   | Max. process+thread count | 3153 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws 
U

[GitHub] [hadoop] hadoop-yetus commented on pull request #3525: HADOOP-17953. S3A: Tests to lookup global or per-bucket configuration for encryption algorithm

[GitBox]

hadoop-yetus commented on pull request #3525:
URL: https://github.com/apache/hadoop/pull/3525#issuecomment-945840872


   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 39s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 3 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 44s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  20m 39s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  21m 13s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  18m 32s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   3m 38s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   2m 35s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 50s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 25s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   3m 53s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  20m 46s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 28s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   1m 35s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  22m  0s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  22m  0s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  19m 26s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  19m 26s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   3m 39s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   2m 23s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m 36s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   2m 17s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   4m  3s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  20m  3s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  17m 32s |  |  hadoop-common in the patch 
passed.  |
   | +1 :green_heart: |  unit  |   2m 35s |  |  hadoop-aws in the patch passed. 
 |
   | +1 :green_heart: |  asflicense  |   1m  0s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 210m 47s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3525/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3525 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux e01a112bdfc6 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 9f2d123c646139feae246076bb61bb91c15dfa9b |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3525/2/testReport/ |
   | Max. process+thread count | 3153 (vs. ulimit of 5500) |
   | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws 
U: . |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3525/2/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above 

[GitHub] [hadoop] jianghuazhu commented on pull request #3544: HDFS-16269.[Fix] Improve NNThroughputBenchmark#blockReport operation.

[GitBox]

jianghuazhu commented on pull request #3544:
URL: https://github.com/apache/hadoop/pull/3544#issuecomment-945826731


   Thanks @jojochuang for the comment.
   Thank you for your reminder. Sorry, I overlooked this point.
   I also found this phenomenon when I tested the online cluster.
   Before the test, there are 3 DN nodes in the online cluster, for example:
   '
   hadoop1..org:50010
   hadoop2..org:50010
   hadoop3..org:50010
   '
   The Xferport of these 3 DN nodes are the same.
   When I try to connect to the online namenode, 3 new DataNode services will 
be created locally, for example:
   ’
   ...:1
   ...:2
   ...:3
   ‘
   When creating a file, you need to get the location of the datanode from the 
remote namenode through addBlock(). At this time, you may get 
hadoop1..org:50010, hadoop2..org:50010, hadoop3..org:50010.
   For the safety of the entire link, I think these DNs should be filtered out 
when executing addBlock().


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?focusedWorklogId=666325&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666325
 ]

ASF GitHub Bot logged work on HADOOP-17965:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 14:18
Start Date: 18/Oct/21 14:18
Worklog Time Spent: 10m 
  Work Description: iwasakims commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-94585


   `./dev-support/bin/create-release` invoking `mvn site` inside worked with 
the patch.
   
   ```
   $ ./dev-support/bin/create-release --docker --dockercache
   ...
   
 Maven Build and Install
   
   
   
   $ /usr/bin/mvn -Dmaven.repo.local=/maven install -Pdist,src,yarn-ui 
-DskipTests -Dtar > /build/source/patchprocess/mvn_install.log 2>&1
   $ /usr/bin/mvn -Dmaven.repo.local=/maven install site site:stage -DskipTests 
-Pdist,src,releasedocs,docs > /build/source/patchprocess/mvn_site.log 2>&1
   ...
   Congratulations, you have successfully built the release
   artifacts for Apache Hadoop 2.10.2-SNAPSHOT
   
   The artifacts for this run are available at 
/home/ubuntu/srcs/hadoop/target/artifacts:
   $ ls -1 /home/ubuntu/srcs/hadoop/target/artifacts
   CHANGES.md
   RELEASENOTES.md
   hadoop-2.10.2-SNAPSHOT-rat.txt
   hadoop-2.10.2-SNAPSHOT-site.tar.gz
   hadoop-2.10.2-SNAPSHOT-src.tar.gz
   hadoop-2.10.2-SNAPSHOT.tar.gz
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666325)
Time Spent: 20m  (was: 10m)

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] iwasakims commented on pull request #3562: HADOOP-17965. Fix documentation build failure using JDK 7 on branch-2.10.

[GitBox]

iwasakims commented on pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562#issuecomment-94585


   `./dev-support/bin/create-release` invoking `mvn site` inside worked with 
the patch.
   
   ```
   $ ./dev-support/bin/create-release --docker --dockercache
   ...
   
 Maven Build and Install
   
   
   
   $ /usr/bin/mvn -Dmaven.repo.local=/maven install -Pdist,src,yarn-ui 
-DskipTests -Dtar > /build/source/patchprocess/mvn_install.log 2>&1
   $ /usr/bin/mvn -Dmaven.repo.local=/maven install site site:stage -DskipTests 
-Pdist,src,releasedocs,docs > /build/source/patchprocess/mvn_site.log 2>&1
   ...
   Congratulations, you have successfully built the release
   artifacts for Apache Hadoop 2.10.2-SNAPSHOT
   
   The artifacts for this run are available at 
/home/ubuntu/srcs/hadoop/target/artifacts:
   $ ls -1 /home/ubuntu/srcs/hadoop/target/artifacts
   CHANGES.md
   RELEASENOTES.md
   hadoop-2.10.2-SNAPSHOT-rat.txt
   hadoop-2.10.2-SNAPSHOT-site.tar.gz
   hadoop-2.10.2-SNAPSHOT-src.tar.gz
   hadoop-2.10.2-SNAPSHOT.tar.gz
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17966) S3A SSE-KMS inconsistency issue during rename

[Steve Loughran (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430030#comment-17430030
 ] 

Steve Loughran commented on HADOOP-17966:
-

Can you grab hadoop 3.3.1 build and see what happens there? I think this is 
what HADOOP-16794 fixed

> S3A SSE-KMS inconsistency issue during rename
> -
>
> Key: HADOOP-17966
> URL: https://issues.apache.org/jira/browse/HADOOP-17966
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common
>Affects Versions: 3.1.2
>Reporter: Dong0829
>Priority: Major
>
> According to the document:
>  
> [https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/encryption.html#S3_Default_Encryption]
>  "Organizations may define a default key in the Amazon KMS; if a default key 
> is set, then it will be used whenever SSE-KMS encryption is chosen and the 
> value of fs.s3a.server-side-encryption.key is empty."
> So basically two conditions to make the object with default KMS: 1. Set 
> SSE-KMS encryption 2. Did not set fs.s3a.server-side-encryption.key
> But there is another confusing scenario below:
> 1. User want to rely on s3 bucket side encryption using their customer KMS 
> key(kms-keyA, for example), so user did not set 
> fs.s3a.server-side-encryption-algorithm or fs.s3a.server-side-encryption.key, 
> and the files uploaded to this bucket will use bucket custom KMS key kms-keyA
>  2. Next step, user want to copy the file to other file using s3a, the 
> process will invoke copyFile() in S3AFileSystem, during the copy, s3a will 
> clone the meta data of the source in cloneObjectMetadata(), in the clone, 
> there is copy of SSE algorithm but no specific kms key copy for the SSE-KMS, 
> it will cause the destination using SSE-KMS without any key id, the final 
> file will use account level default key under aws/s3(
>  [https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object.html),]
>  lets say its kms-keyB.
> It means when ever there is a copy, the kms key will be changed from customer 
> key kms-keyA to kms-keyB, which will cause inconsistency, for example:
> hdfs dfs -put test s3://ssetest/
> During this put, there will be rename processing from test.__COPYING__ to 
> test, it will cause the final test file encrypted with account default key 
> kms-keyB instead of s3 bucket customer key kms-keyA which is expected. Should 
> we consider to clone the KMS key id also to keep the consistency?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-17559) S3Guard import can OOM on large imports

[Steve Loughran (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HADOOP-17559.
-
Resolution: Won't Fix

> S3Guard import can OOM on large imports
> ---
>
> Key: HADOOP-17559
> URL: https://issues.apache.org/jira/browse/HADOOP-17559
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: 3.3.1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> I know I'm closing ~all S3Guard issues as wontfix, but this is pressing so 
> I'm going to do it anyway
> S3guard import of directory tree containing many, many files will OOM. 
> Looking at the code this is going to be because
> * import tool builds a map of all dirs imported, which as the comments note 
> "superfluous for DDB". - *cut*
> * DDB AncestorState tracks files as well as dirs, purely as a safety check to 
> make sure current op doesn't somehow write a file entry above a dir entry in 
> the same operation
> We've been running S3Guard for a long time, and condition #2 has never arisen.
> Propose: don't store filenames there, so memory consumption goes from O(files 
> + dirs) to O(dirs)
> Code straightforward, can't think of any tests



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-16646) Backport S3A enhancements and fixes from trunk to branch-3.2

[Steve Loughran (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HADOOP-16646.
-
Resolution: Won't Fix

> Backport S3A enhancements and fixes from trunk to branch-3.2
> 
>
> Key: HADOOP-16646
> URL: https://issues.apache.org/jira/browse/HADOOP-16646
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs, fs/s3
>Affects Versions: 3.2.1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Backport all the stable features from hadoop-aws in trunk to branch-3.2
> Note. we've already pulled most of these into CDP, so they have had 
> integration testing already done, though there may be some differences in 
> dependencies (guava, mockito etc)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work stopped] (HADOOP-16646) Backport S3A enhancements and fixes from trunk to branch-3.2

[Steve Loughran (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HADOOP-16646 stopped by Steve Loughran.
---
> Backport S3A enhancements and fixes from trunk to branch-3.2
> 
>
> Key: HADOOP-16646
> URL: https://issues.apache.org/jira/browse/HADOOP-16646
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: fs, fs/s3
>Affects Versions: 3.2.1
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Backport all the stable features from hadoop-aws in trunk to branch-3.2
> Note. we've already pulled most of these into CDP, so they have had 
> integration testing already done, though there may be some differences in 
> dependencies (guava, mockito etc)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HADOOP-17965:

Labels: pull-request-available  (was: )

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?focusedWorklogId=666295&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666295
 ]

ASF GitHub Bot logged work on HADOOP-17965:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 13:31
Start Date: 18/Oct/21 13:31
Worklog Time Spent: 10m 
  Work Description: iwasakims opened a new pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562


   https://issues.apache.org/jira/browse/HADOOP-17965
   
   `mvn site` by JDK 7 fails due to error related to spotbugs which does not 
support Java 7.
   
   ```
   $ ./start-build-env.sh
   ubuntu@2a660195e110:~/hadoop$ mvn clean install -DskipTests
   ubuntu@2a660195e110:~/hadoop$ mvn site
   ...(snip)
   [INFO] Apache Hadoop YARN Site  SUCCESS [  2.138 
s]
   [INFO] Apache Hadoop YARN UI .. SUCCESS [  0.044 
s]
   [INFO] Apache Hadoop YARN Project . FAILURE [  0.700 
s]
   ...(snip)
   [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project 
hadoop-yarn-project: Execution default-site of goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API 
incompatibility was encountered while executing 
org.apache.maven.plugins:maven-site-plugin:3.5:site: 
java.lang.UnsupportedClassVersionError: 
org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported major.minor 
version 52.0
   ```
   
   Since we are using JDK 7 in the Dockerfile, dev-support/bin/create-release 
is affected too.
   
   The cause is obsolete  entries of pom.xml for already resolved 
[MSITE-443](https://issues.apache.org/jira/browse/MSITE-443).
   
   We can safely remove the  entry. `mvn site` generates same 
contents without the entry. spotbugsXml.xml are generated regardless of it (on 
`mvn compile spotbugs:spotbugs` using JDK 8).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666295)
Remaining Estimate: 0h
Time Spent: 10m

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] iwasakims opened a new pull request #3562: HADOOP-17965. Fix documentation build failure using JDK 7 on branch-2.10.

[GitBox]

iwasakims opened a new pull request #3562:
URL: https://github.com/apache/hadoop/pull/3562


   https://issues.apache.org/jira/browse/HADOOP-17965
   
   `mvn site` by JDK 7 fails due to error related to spotbugs which does not 
support Java 7.
   
   ```
   $ ./start-build-env.sh
   ubuntu@2a660195e110:~/hadoop$ mvn clean install -DskipTests
   ubuntu@2a660195e110:~/hadoop$ mvn site
   ...(snip)
   [INFO] Apache Hadoop YARN Site  SUCCESS [  2.138 
s]
   [INFO] Apache Hadoop YARN UI .. SUCCESS [  0.044 
s]
   [INFO] Apache Hadoop YARN Project . FAILURE [  0.700 
s]
   ...(snip)
   [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project 
hadoop-yarn-project: Execution default-site of goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API 
incompatibility was encountered while executing 
org.apache.maven.plugins:maven-site-plugin:3.5:site: 
java.lang.UnsupportedClassVersionError: 
org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported major.minor 
version 52.0
   ```
   
   Since we are using JDK 7 in the Dockerfile, dev-support/bin/create-release 
is affected too.
   
   The cause is obsolete  entries of pom.xml for already resolved 
[MSITE-443](https://issues.apache.org/jira/browse/MSITE-443).
   
   We can safely remove the  entry. `mvn site` generates same 
contents without the entry. spotbugsXml.xml are generated regardless of it (on 
`mvn compile spotbugs:spotbugs` using JDK 8).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HADOOP-16082:

Labels: pull-request-available  (was: )

> FsShell ls: Add option -i to print inode id
> ---
>
> Key: HADOOP-16082
> URL: https://issues.apache.org/jira/browse/HADOOP-16082
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: common
>Affects Versions: 3.2.0, 3.1.1
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
>  Labels: pull-request-available
> Attachments: HADOOP-16082.001.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When debugging the FSImage corruption issue, I often need to know a file's or 
> directory's inode id. At this moment, the only way to do that is to use OIV 
> tool to dump the FSImage and look up the filename, which is very inefficient.
> Here I propose adding option "-i" in FsShell that prints files' or 
> directories' inode id.
> h2. Implementation
> h3. For hdfs:// (HDFS)
> fileId exists in HdfsLocatedFileStatus, which is already returned to 
> hdfs-client. We just need to print it in Ls#processPath().
> h3. For file:// (Local FS)
> h4. Linux
> Use java.nio.
> h4. Windows
> Windows has the concept of "File ID" which is similar to inode id. It is 
> unique in NTFS and ReFS.
> h3. For other FS
> The fileId entry will be "0" in FileStatus if it is not set. We could either 
> ignore or throw an exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-16082) FsShell ls: Add option -i to print inode id

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-16082?focusedWorklogId=666292&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666292
 ]

ASF GitHub Bot logged work on HADOOP-16082:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 13:26
Start Date: 18/Oct/21 13:26
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #1271:
URL: https://github.com/apache/hadoop/pull/1271#issuecomment-945766138


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  12m 20s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 51s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  20m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  21m 24s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  18m 42s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   3m 39s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   4m 24s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   3m 14s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   4m 18s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   8m 26s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 52s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  20m 41s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  20m 41s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  18m 39s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  18m 39s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   3m 36s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   4m 19s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   3m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   4m 12s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   8m 55s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m  7s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  17m 23s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   2m 43s |  |  hadoop-hdfs-client in the patch 
passed.  |
   | -1 :x: |  unit  | 227m 15s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 13s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 469m  7s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.cli.TestCLI |
   |   | hadoop.hdfs.web.TestJsonUtil |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/1271 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux dd20b22ee00b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git 

[GitHub] [hadoop] hadoop-yetus commented on pull request #1271: HADOOP-16082. FsShell ls: Add option -i to print inode id

[GitBox]

hadoop-yetus commented on pull request #1271:
URL: https://github.com/apache/hadoop/pull/1271#issuecomment-945766138


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |  12m 20s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 1 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +0 :ok: |  mvndep  |  12m 51s |  |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  20m 40s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |  21m 24s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |  18m 42s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   3m 39s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   4m 24s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   3m 14s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   4m 18s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   8m 26s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  21m 52s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 27s |  |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   2m 59s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  20m 41s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |  20m 41s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |  18m 39s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |  18m 39s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   3m 36s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   4m 19s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   3m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   4m 12s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   8m 55s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m  7s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  17m 23s | 
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
 |  hadoop-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   2m 43s |  |  hadoop-hdfs-client in the patch 
passed.  |
   | -1 :x: |  unit  | 227m 15s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   1m 13s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 469m  7s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.cli.TestCLI |
   |   | hadoop.hdfs.web.TestJsonUtil |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/1271 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux dd20b22ee00b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 717a5a7a8056b4fddb3140730d01f00adcb64f6a |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1271/1/testReport/ |
  

[jira] [Commented] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430010#comment-17430010
 ] 

Masatake Iwasaki commented on HADOOP-17965:
---

The  entry seems to be for already resolved MSITE-443.

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1742#comment-1742
 ] 

Masatake Iwasaki edited comment on HADOOP-17965 at 10/18/21, 1:15 PM:
--

The following entry of hadoop-yarn-project/pom.xml seems to be the cause. I got 
no issue on sub-modules other than hadoop-yarn-project and 
hadoop-mapreduce-project on {{mvn site}}.
{noformat}
  

  
com.github.spotbugs
spotbugs-maven-plugin

  true

  

  
{noformat}


was (Author: iwasakims):
The following entry of hadoop-yarn-project/pom.xml seems to be the cause. I got 
no issue on other sub-modules on {{mvn site}}.

{noformat}
  

  
com.github.spotbugs
spotbugs-maven-plugin

  true

  

  
{noformat}

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430006#comment-17430006
 ] 

Masatake Iwasaki edited comment on HADOOP-17965 at 10/18/21, 1:08 PM:
--

I think we can safely remove the  entry. spotbugsXml.xml are 
generated regardless of the entry (using JDK 8). {{mvn site}} generated same 
contents.
{noformat}
$ mvn compile spotbugs:spotbugs
$ find . -name spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-httpfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-nfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-client/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-rbf/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-sharedcachemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-1/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/target/spotbugsXml.xml
./hadoop-tools/hadoop-archives/target/spotbugsXml.xml
./hadoop-tools/hadoop-sls/target/spotbugsXml.xml
./hadoop-tools/hadoop-ant/target/spotbugsXml.xml
./hadoop-tools/hadoop-openstack/target/spotbugsXml.xml
./hadoop-tools/hadoop-rumen/target/spotbugsXml.xml
./hadoop-tools/hadoop-datajoin/target/spotbugsXml.xml
./hadoop-tools/hadoop-azure-datalake/target/spotbugsXml.xml
./hadoop-tools/hadoop-archive-logs/target/spotbugsXml.xml
./hadoop-tools/hadoop-aliyun/target/spotbugsXml.xml
./hadoop-tools/hadoop-resourceestimator/target/spotbugsXml.xml
./hadoop-tools/hadoop-extras/target/spotbugsXml.xml
./hadoop-tools/hadoop-gridmix/target/spotbugsXml.xml
./hadoop-tools/hadoop-aws/target/spotbugsXml.xml
./hadoop-tools/hadoop-streaming/target/spotbugsXml.xml
./hadoop-tools/hadoop-azure/target/spotbugsXml.xml
./hadoop-tools/hadoop-distcp/target/spotbugsXml.xml
./hadoop-common-project/hadoop-auth/target/spotbugsXml.xml
./hadoop-common-project/hadoop-auth-examples/target/spotbugsXml.xml
./hadoop-common-project/hadoop-nfs/target/spotbugsXml.xml
./hadoop-common-project/hadoop-annotations/target/spotbugsXml.xml
./hadoop-common-project/hadoop-kms/target/spotbugsXml.xml
./hadoop-common-project/hadoop-minikdc/target/spotbugsXml.xml
./hadoop-common-project/hadoop-common/target/spotbugsXml.xml
./hadoop-maven-plugins/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs-plugins/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/spotbugs

[jira] [Commented] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430006#comment-17430006
 ] 

Masatake Iwasaki commented on HADOOP-17965:
---

I think we can safely remove the  entry. spotbugsXml.xml are 
generated regardless of the entry. {{mvn site}} generated same contents.

{noformat}
$ mvn compile spotbugs:spotbugs
$ find . -name spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-httpfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-nfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-client/target/spotbugsXml.xml
./hadoop-hdfs-project/hadoop-hdfs-rbf/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-sharedcachemanager/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-1/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/spotbugsXml.xml
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/target/spotbugsXml.xml
./hadoop-tools/hadoop-archives/target/spotbugsXml.xml
./hadoop-tools/hadoop-sls/target/spotbugsXml.xml
./hadoop-tools/hadoop-ant/target/spotbugsXml.xml
./hadoop-tools/hadoop-openstack/target/spotbugsXml.xml
./hadoop-tools/hadoop-rumen/target/spotbugsXml.xml
./hadoop-tools/hadoop-datajoin/target/spotbugsXml.xml
./hadoop-tools/hadoop-azure-datalake/target/spotbugsXml.xml
./hadoop-tools/hadoop-archive-logs/target/spotbugsXml.xml
./hadoop-tools/hadoop-aliyun/target/spotbugsXml.xml
./hadoop-tools/hadoop-resourceestimator/target/spotbugsXml.xml
./hadoop-tools/hadoop-extras/target/spotbugsXml.xml
./hadoop-tools/hadoop-gridmix/target/spotbugsXml.xml
./hadoop-tools/hadoop-aws/target/spotbugsXml.xml
./hadoop-tools/hadoop-streaming/target/spotbugsXml.xml
./hadoop-tools/hadoop-azure/target/spotbugsXml.xml
./hadoop-tools/hadoop-distcp/target/spotbugsXml.xml
./hadoop-common-project/hadoop-auth/target/spotbugsXml.xml
./hadoop-common-project/hadoop-auth-examples/target/spotbugsXml.xml
./hadoop-common-project/hadoop-nfs/target/spotbugsXml.xml
./hadoop-common-project/hadoop-annotations/target/spotbugsXml.xml
./hadoop-common-project/hadoop-kms/target/spotbugsXml.xml
./hadoop-common-project/hadoop-minikdc/target/spotbugsXml.xml
./hadoop-common-project/hadoop-common/target/spotbugsXml.xml
./hadoop-maven-plugins/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs-plugins/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/spotbugsXml.xml
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-m

[jira] [Commented] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1742#comment-1742
 ] 

Masatake Iwasaki commented on HADOOP-17965:
---

The following entry of hadoop-yarn-project/pom.xml seems to be the cause. I got 
no issue on other sub-modules on {{mvn site}}.

{noformat}
  

  
com.github.spotbugs
spotbugs-maven-plugin

  true

  

  
{noformat}

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] rainforc closed pull request #71: fix NullPointerException in Balancer

[GitBox]

rainforc closed pull request #71:
URL: https://github.com/apache/hadoop/pull/71


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

[ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429990#comment-17429990
 ] 

Masatake Iwasaki commented on HADOOP-17965:
---

{noformat}
$ ./start-build-env.sh
ubuntu@2a660195e110:~/hadoop$ mvn clean install -DskipTests
ubuntu@2a660195e110:~/hadoop$ mvn site
...(snip)
[INFO] Apache Hadoop YARN Site  SUCCESS [  2.138 s]
[INFO] Apache Hadoop YARN UI .. SUCCESS [  0.044 s]
[INFO] Apache Hadoop YARN Project . FAILURE [  0.700 s]
...(snip)
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project 
hadoop-yarn-project: Execution default-site of goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API 
incompatibility was encountered while executing 
org.apache.maven.plugins:maven-site-plugin:3.5:site: 
java.lang.UnsupportedClassVersionError: 
org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported major.minor 
version 52.0
{noformat}

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Masatake Iwasaki updated HADOOP-17965:
--
Description: {{mvn site}} by JDK 7 fails due to error related to spotbugs 
which does not support Java 7.  (was: {noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project 
hadoop-yarn-project: Execution default-site of goal 
org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API 
incompatibility was encountered while executing 
org.apache.maven.plugins:maven-site-plugin:3.5:site: 
java.lang.UnsupportedClassVersionError: 
org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported major.minor 
version 52.0
{noformat}
)

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {{mvn site}} by JDK 7 fails due to error related to spotbugs which does not 
> support Java 7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17965) Fix documentation build failure using JDK 7 on branch-2.10

[Masatake Iwasaki (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Masatake Iwasaki updated HADOOP-17965:
--
Summary: Fix documentation build failure using JDK 7 on branch-2.10  (was: 
Fix documentation build failure using create-release on branch-2.10)

> Fix documentation build failure using JDK 7 on branch-2.10
> --
>
> Key: HADOOP-17965
> URL: https://issues.apache.org/jira/browse/HADOOP-17965
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: build
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Major
>
> {noformat}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-site-plugin:3.5:site (default-site) on project 
> hadoop-yarn-project: Execution default-site of goal 
> org.apache.maven.plugins:maven-site-plugin:3.5:site failed: An API 
> incompatibility was encountered while executing 
> org.apache.maven.plugins:maven-site-plugin:3.5:site: 
> java.lang.UnsupportedClassVersionError: 
> org/codehaus/mojo/spotbugs/SpotbugsViolationCheckMojo : Unsupported 
> major.minor version 52.0
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] shuzirra commented on a change in pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

shuzirra commented on a change in pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#discussion_r730821045



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
##
@@ -2326,6 +2326,57 @@ public void stopQueue() {
 }
   }
 
+  void updateMaximumApplications(CapacitySchedulerConfiguration conf) {
+int maxAppsForQueue = conf.getMaximumApplicationsPerQueue(getQueuePath());
+
+int maxGlobalApplications = conf.getGlobalMaximumApplicationsPerQueue();
+int maxSystemApplications = conf.getMaximumSystemApplications();
+int baseMaxApplications = maxGlobalApplications > 0 ?
+maxGlobalApplications : maxSystemApplications;

Review comment:
   According to the documentation:
   Maximum number of applications in the system which can be concurrently 
active both running and pending. Limits on each queue are directly proportional 
to their queue capacities and user limits. This is a hard limit and any 
applications submitted when this limit is reached will be rejected. Default is 
1
   This means to me, that maxSystemApp should be an absolute upper limit, if 
the user wishes to go above it, they should raise this limit. MaxGlobalApp 
should be always <= maxSystem app. But look at this the following way: if the 
user has 100 queues with maxGlobalApp 1000, and they saturate 10 of them with 
1 applications, then they won't be able to start any application in the 
11th queue, because of the maxSystem is reached. So this limitation is already 
in place. As it should be. 
   See: 
https://github.com/apache/hadoop/blob/5337bebcc53a0144311877362787ed9ece0cb118/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java#L1044
   
   So capacity scheduler will enforce the max system application anyway, even 
if the leaf queue would allow it, so there is no point in allowing more 
application per leaf queue than the system wide max applications.
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] lujiefsi edited a comment on pull request #3561: YARN-10980:fix CVE-2020-8908

[GitBox]

lujiefsi edited a comment on pull request #3561:
URL: https://github.com/apache/hadoop/pull/3561#issuecomment-945645510


   due to forget to fetech upstream, i open this issue. this bug is fixed by 
https://issues.apache.org/jira/browse/HADOOP-17653. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka commented on pull request #3524: HDFS-16257. Set initialCapacity for guava cache to solve performance issue

[GitBox]

aajisaka commented on pull request #3524:
URL: https://github.com/apache/hadoop/pull/3524#issuecomment-945649327


   Thank you @symious for your contribution and thanks @goiri for your review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka merged pull request #3524: HDFS-16257. Set initialCapacity for guava cache to solve performance issue

[GitBox]

aajisaka merged pull request #3524:
URL: https://github.com/apache/hadoop/pull/3524


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] sodonnel merged pull request #3548: HDFS-16272. Fix int overflow in computing safe length during EC block recovery

[GitBox]

sodonnel merged pull request #3548:
URL: https://github.com/apache/hadoop/pull/3548


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] 9uapaw commented on a change in pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

9uapaw commented on a change in pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#discussion_r730802520



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
##
@@ -2326,6 +2326,57 @@ public void stopQueue() {
 }
   }
 
+  void updateMaximumApplications(CapacitySchedulerConfiguration conf) {
+int maxAppsForQueue = conf.getMaximumApplicationsPerQueue(getQueuePath());
+
+int maxGlobalApplications = conf.getGlobalMaximumApplicationsPerQueue();
+int maxSystemApplications = conf.getMaximumSystemApplications();
+int baseMaxApplications = maxGlobalApplications > 0 ?
+maxGlobalApplications : maxSystemApplications;

Review comment:
   The problem with this approach is that system max app defaults to 1. 
This means you basically can not set maxGlobalApp higher than this value, 
because it will be trimmed to 1. We could pursue this path by making 
maxSystemApp defaults to -1 and check if it is defined, but I am not convinced 
it is worth the effort.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] lujiefsi commented on pull request #3561: Yarn 10980

[GitBox]

lujiefsi commented on pull request #3561:
URL: https://github.com/apache/hadoop/pull/3561#issuecomment-945645510


   due to forget to fetech upstream, i open this issue. this bug is fixed by 
https://issues.apache.org/jira/browse/HADOOP-17963. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] lujiefsi closed pull request #3561: Yarn 10980

[GitBox]

lujiefsi closed pull request #3561:
URL: https://github.com/apache/hadoop/pull/3561


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] lujiefsi opened a new pull request #3561: Yarn 10980

[GitBox]

lujiefsi opened a new pull request #3561:
URL: https://github.com/apache/hadoop/pull/3561


   see  https://www.cvedetails.com/cve/CVE-2020-8908/
   

   
   A temp directory creation vulnerability exists in all versions of Guava, 
allowing an attacker with access to the machine to potentially access data in a 
temporary directory created by the Guava API 
com.google.common.io.Files.createTempDir(). By default, on unix-like systems, 
the created directory is world-readable (readable by an attacker with access to 
the system). The method in question has been marked @Deprecated in versions 
30.0 and later and should not be used. For Android developers, we recommend 
choosing a temporary directory API provided by Android, such as 
context.getCacheDir(). For other Java developers, we recommend migrating to the 
Java 7 API java.nio.file.Files.createTempDirectory() which explicitly 
configures permissions of 700, or configuring the Java runtime's java.io.tmpdir 
system property to point to a location whose permissions are appropriately 
configured.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] tomscut edited a comment on pull request #3538: HDFS-16266. Add remote port information to HDFS audit log

[GitBox]

tomscut edited a comment on pull request #3538:
URL: https://github.com/apache/hadoop/pull/3538#issuecomment-944128111


   Thanks @Akira Ajisaka ***@***.***> for your comments and
   suggestion. I will update it ASAP.
   
   Akira Ajisaka ***@***.***> 于2021年10月15日周五 下午3:27写道：
   
   > And we can make it optional as @iwasakims 
   > suggested. Thank you very much.
   >
   > +1 to make it optional.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > , or
   > unsubscribe
   > 

   > .
   > Triage notifications on the go with GitHub Mobile for iOS
   > 

   > or Android
   > 
.
   >
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17932) Distcp file length comparison have no effect

[Akira Ajisaka (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka updated HADOOP-17932:
---
Summary: Distcp file length comparison have no effect  (was: distcp file 
length comparison have no effect)

> Distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Assignee: yinan zhan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-17932) Distcp file length comparison have no effect

[Akira Ajisaka (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka updated HADOOP-17932:
---
Description: 
the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand have 
no effect

current is
{code:java}
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
targetPath, skipCrc, source.getLen());{code}
{code:java}
public static void compareFileLengthsAndChecksums(long srcLen,
   FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
   FileSystem targetFS, Path target, boolean skipCrc,
   long targetLen) throws IOException {
  if (srcLen != targetLen) {
throw new IOException(
DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
+ ") and target:" + target + " (" + targetLen + ")");
  }
{code}
so, compare source.getLen() with source.getLen()...

It should be like below in history view
{code:java}
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
targetPath, skipCrc, offset + bytesRead);
{code}
 

 

  was:
the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand have 
no effect

current is
{code:java}
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
targetPath, skipCrc, source.getLen());{code}
{code:java}
public static void compareFileLengthsAndChecksums(long srcLen,
   FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
   FileSystem targetFS, Path target, boolean skipCrc,
   long targetLen) throws IOException {
  if (srcLen != targetLen) {
throw new IOException(
DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
+ ") and target:" + target + " (" + targetLen + ")");
  }
{code}
so, compare source.getLen() with source.getLen()...

It should be like below in history view
{code:java}
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
targetPath, skipCrc, bytesRead);
{code}
 

 


> Distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Assignee: yinan zhan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, offset + bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Assigned] (HADOOP-17932) distcp file length comparison have no effect

[Akira Ajisaka (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka reassigned HADOOP-17932:
--

Assignee: yinan zhan

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Assignee: yinan zhan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-17932) distcp file length comparison have no effect

[Akira Ajisaka (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka resolved HADOOP-17932.

Fix Version/s: 3.3.2
   3.4.0
   Resolution: Fixed

Committed to trunk and branch-3.3. Thanks [~adol] for your contribution!

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0, 3.3.2
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17932) distcp file length comparison have no effect

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666204&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666204
 ]

ASF GitHub Bot logged work on HADOOP-17932:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 10:08
Start Date: 18/Oct/21 10:08
Worklog Time Spent: 10m 
  Work Description: aajisaka merged pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666204)
Time Spent: 1h 10m  (was: 1h)

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17932) distcp file length comparison have no effect

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666205&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666205
 ]

ASF GitHub Bot logged work on HADOOP-17932:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 10:08
Start Date: 18/Oct/21 10:08
Worklog Time Spent: 10m 
  Work Description: aajisaka commented on pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#issuecomment-945612503


   Merged. Thank you @adol001 and @jojochuang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666205)
Time Spent: 1h 20m  (was: 1h 10m)

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka commented on pull request #3519: HADOOP-17932. Distcp file length comparison have no effect

[GitBox]

aajisaka commented on pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#issuecomment-945612503


   Merged. Thank you @adol001 and @jojochuang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka merged pull request #3519: HADOOP-17932. Distcp file length comparison have no effect

[GitBox]

aajisaka merged pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17888) The error of Constant annotation in AzureNativeFileSystemStore.java

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17888?focusedWorklogId=666203&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666203
 ]

ASF GitHub Bot logged work on HADOOP-17888:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 10:07
Start Date: 18/Oct/21 10:07
Worklog Time Spent: 10m 
  Work Description: steveloughran commented on pull request #3372:
URL: https://github.com/apache/hadoop/pull/3372#issuecomment-945611957


   My laptop died last week. Still don't have a work laptop; this is my windows 
one which can't actually do any hadoop dev as the paths in the hadoop trunk 
source tree are too long for the OS. Just starting back on reviewing things


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666203)
Time Spent: 1h 10m  (was: 1h)

> The error of Constant  annotation in AzureNativeFileSystemStore.java
> 
>
> Key: HADOOP-17888
> URL: https://issues.apache.org/jira/browse/HADOOP-17888
> Project: Hadoop Common
>  Issue Type: Improvement
>Reporter: guoxin
>Assignee: guoxin
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #3372: HADOOP-17888. The error of Constant annotation in AzureNativeFileSystem…

[GitBox]

steveloughran commented on pull request #3372:
URL: https://github.com/apache/hadoop/pull/3372#issuecomment-945611957


   My laptop died last week. Still don't have a work laptop; this is my windows 
one which can't actually do any hadoop dev as the paths in the hadoop trunk 
source tree are too long for the OS. Just starting back on reviewing things


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] hadoop-yetus commented on pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

hadoop-yetus commented on pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#issuecomment-945610396


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 40s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  31m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m  2s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 54s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 48s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 59s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 48s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 43s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 50s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  19m 53s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 49s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 52s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   0m 52s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 45s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   0m 45s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 39s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 39s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 35s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   1m 52s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  19m 36s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  |  95m 43s | 
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
 |  hadoop-yarn-server-resourcemanager in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 34s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 181m 37s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | 
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestParentQueue |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/5/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3500 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux d59351016675 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / cb268008137b6e5b1b1ab2f1be518efe17fa99c7 |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3500/5/testReport/ |
   | Max. process+thread count | 983 (vs. ulimit of 5500) |
   | modules | C: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
 U: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resour

[GitHub] [hadoop] 9uapaw commented on a change in pull request #3551: YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue or group fields by feature if possible

[GitBox]

9uapaw commented on a change in pull request #3551:
URL: https://github.com/apache/hadoop/pull/3551#discussion_r730729218



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueNodeLabelsSettings.java
##
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.util.Sets;
+import 
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager;
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.ROOT;
+import java.io.IOException;
+import java.util.Set;
+
+public class QueueNodeLabelsSettings {
+  private final CSQueue parent;
+  private final String queuePath;
+  private final CapacitySchedulerContext csContext;
+  private Set accessibleLabels;
+  private Set configuredNodeLabels;
+  private String defaultLabelExpression;
+
+  public QueueNodeLabelsSettings(CapacitySchedulerConfiguration configuration,
+  CSQueue parent,
+  String queuePath,

Review comment:
   See the comment below (let us incorporate the new QueuePath class).

##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/QueueNodeLabelsSettings.java
##
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.util.Sets;
+import 
org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager;
+import static 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.ROOT;
+import java.io.IOException;
+import java.util.Set;
+
+public class QueueNodeLabelsSettings {

Review comment:
   Can you describe briefly what this class is used for?

##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/BasicQueueProperties.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity;
+
+public class BasicQueueProperties {
+  private final String queueName;
+  private final String queuePath;
+  private CSQueue parent;
+
+  public BasicQueueP

[jira] [Work logged] (HADOOP-17932) distcp file length comparison have no effect

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666193&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666193
 ]

ASF GitHub Bot logged work on HADOOP-17932:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 09:50
Start Date: 18/Oct/21 09:50
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#issuecomment-945597422


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 39s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  31m 12s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 27s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 36s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   0m 50s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  18m 45s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 26s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 22s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   0m 22s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 16s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 25s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 21s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  18m 28s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  21m 26s |  |  hadoop-distcp in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  99m 50s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3519 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 3b4de3e9b602 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / a714dd781865d8fe5d1c69dbde105dbd20e6a21e |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/testReport/ |
   | Max. process+thread count | 693 (vs. ulimit of 5500) |
   | modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/console |
   | versions | git=2.25.

[GitHub] [hadoop] hadoop-yetus commented on pull request #3519: HADOOP-17932. Distcp file length comparison have no effect

[GitBox]

hadoop-yetus commented on pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#issuecomment-945597422


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 39s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | -1 :x: |  test4tests  |   0m  0s |  |  The patch doesn't appear to include 
any new or modified tests. Please justify why no new tests are needed for this 
patch. Also please list what manual steps were performed to verify this patch.  
|
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  31m 12s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   0m 33s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   0m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   0m 27s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   0m 36s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 31s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 28s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   0m 50s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  18m 45s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   0m 27s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 26s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   0m 26s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   0m 22s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   0m 22s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 16s |  |  the patch passed  |
   | +1 :green_heart: |  mvnsite  |   0m 25s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 21s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   0m 19s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   0m 50s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  18m 28s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | +1 :green_heart: |  unit  |  21m 26s |  |  hadoop-distcp in the patch 
passed.  |
   | +1 :green_heart: |  asflicense  |   0m 35s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   |  99m 50s |  |  |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3519 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux 3b4de3e9b602 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / a714dd781865d8fe5d1c69dbde105dbd20e6a21e |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/testReport/ |
   | Max. process+thread count | 693 (vs. ulimit of 5500) |
   | modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp |
   | Console output | 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3519/2/console |
   | versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
   | Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this servic

[jira] [Work logged] (HADOOP-17932) distcp file length comparison have no effect

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666176&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666176
 ]

ASF GitHub Bot logged work on HADOOP-17932:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 08:26
Start Date: 18/Oct/21 08:26
Worklog Time Spent: 10m 
  Work Description: adol001 commented on a change in pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#discussion_r730684343



##
File path: 
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##
@@ -154,7 +154,7 @@ private long doCopy(CopyListingFileStatus source, Path 
target,
   if (!source.isSplit()) {
 DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
 sourcePath, sourceChecksum, targetFS,
-targetPath, skipCrc, source.getLen());
+targetPath, skipCrc, bytesRead);

Review comment:
   "
 compareFileLengths(sourceFileStatus, targetPath, configuration, 
bytesRead
 + offset);
   "
   When I copied from old version hadoop, I encountered a line break...Thank you




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666176)
Time Spent: 50m  (was: 40m)

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] adol001 commented on a change in pull request #3519: HADOOP-17932. Distcp file length comparison have no effect

[GitBox]

adol001 commented on a change in pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#discussion_r730684343



##
File path: 
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##
@@ -154,7 +154,7 @@ private long doCopy(CopyListingFileStatus source, Path 
target,
   if (!source.isSplit()) {
 DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
 sourcePath, sourceChecksum, targetFS,
-targetPath, skipCrc, source.getLen());
+targetPath, skipCrc, bytesRead);

Review comment:
   "
 compareFileLengths(sourceFileStatus, targetPath, configuration, 
bytesRead
 + offset);
   "
   When I copied from old version hadoop, I encountered a line break...Thank you




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Work logged] (HADOOP-17932) distcp file length comparison have no effect

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666163&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666163
 ]

ASF GitHub Bot logged work on HADOOP-17932:
---

Author: ASF GitHub Bot
Created on: 18/Oct/21 07:58
Start Date: 18/Oct/21 07:58
Worklog Time Spent: 10m 
  Work Description: aajisaka commented on a change in pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#discussion_r730659799



##
File path: 
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##
@@ -154,7 +154,7 @@ private long doCopy(CopyListingFileStatus source, Path 
target,
   if (!source.isSplit()) {
 DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
 sourcePath, sourceChecksum, targetFS,
-targetPath, skipCrc, source.getLen());
+targetPath, skipCrc, bytesRead);

Review comment:
   The test failures are related to the change. Probably we need to add 
offset.
   ```suggestion
   targetPath, skipCrc, offset + bytesRead);
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 666163)
Time Spent: 40m  (was: 0.5h)

> distcp file length comparison have no effect
> 
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, tools, tools/distcp
>Affects Versions: 3.3.1
>Reporter: yinan zhan
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand 
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
>FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
>FileSystem targetFS, Path target, boolean skipCrc,
>long targetLen) throws IOException {
>   if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
>   }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] aajisaka commented on a change in pull request #3519: HADOOP-17932. Distcp file length comparison have no effect

[GitBox]

aajisaka commented on a change in pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#discussion_r730659799



##
File path: 
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##
@@ -154,7 +154,7 @@ private long doCopy(CopyListingFileStatus source, Path 
target,
   if (!source.isSplit()) {
 DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
 sourcePath, sourceChecksum, targetFS,
-targetPath, skipCrc, source.getLen());
+targetPath, skipCrc, bytesRead);

Review comment:
   The test failures are related to the change. Probably we need to add 
offset.
   ```suggestion
   targetPath, skipCrc, offset + bytesRead);
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] 9uapaw commented on a change in pull request #3500: YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and fin…

[GitBox]

9uapaw commented on a change in pull request #3500:
URL: https://github.com/apache/hadoop/pull/3500#discussion_r730622413



##
File path: 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
##
@@ -2326,6 +2326,57 @@ public void stopQueue() {
 }
   }
 
+  void updateMaximumApplications(CapacitySchedulerConfiguration conf) {
+int maxAppsForQueue = conf.getMaximumApplicationsPerQueue(getQueuePath());
+
+int maxGlobalApplications = conf.getGlobalMaximumApplicationsPerQueue();
+int maxSystemApplications = conf.getMaximumSystemApplications();
+int baseMaxApplications = maxGlobalApplications > 0 ?
+maxGlobalApplications : maxSystemApplications;

Review comment:
   I agree on this and this change still conforms to our existing tests.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org