[jira] [Updated] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
[
https://issues.apache.org/jira/browse/HADOOP-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ranith Sardar updated HADOOP-16585:
---
Attachment: (was: HADOOP-16585.002.patch)
> [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating
> file LoadGenerator#write
> ---
>
> Key: HADOOP-16585
> URL: https://issues.apache.org/jira/browse/HADOOP-16585
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ranith Sardar
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HADOOP-16585.001.patch
>
>
> {code:java}
> // id would be same for multiple file, so it may occur file not found
> exception
> private void write() throws IOException {
> String dirName = dirs.get(r.nextInt(dirs.size()));
> Path file = new Path(dirName, hostname+id);
>
> fc.delete(file, true);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1711: Hadoop 16455
hadoop-yetus commented on issue #1711: Hadoop 16455 URL: https://github.com/apache/hadoop/pull/1711#issuecomment-553270481 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1092 | trunk passed | | +1 | compile | 32 | trunk passed | | +1 | checkstyle | 25 | trunk passed | | +1 | mvnsite | 35 | trunk passed | | +1 | shadedclient | 796 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 49 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 28 | the patch passed | | +1 | compile | 25 | the patch passed | | +1 | javac | 25 | the patch passed | | -0 | checkstyle | 18 | hadoop-tools/hadoop-azure: The patch generated 3 new + 6 unchanged - 1 fixed = 9 total (was 7) | | +1 | mvnsite | 27 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 801 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | the patch passed | | -1 | findbugs | 57 | hadoop-tools/hadoop-azure generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 83 | hadoop-azure in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 3305 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-tools/hadoop-azure | | | Dead store to e in org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.throwFileNotFoundIfNotExists(String) At AzureBlobFileSystemStore.java:org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.throwFileNotFoundIfNotExists(String) At AzureBlobFileSystemStore.java:[line 288] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1711 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cf7a5a1e0614 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/artifact/out/new-findbugs-hadoop-tools_hadoop-azure.html | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/testReport/ | | Max. process+thread count | 411 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
[
https://issues.apache.org/jira/browse/HADOOP-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973066#comment-16973066
]
Ranith Sardar commented on HADOOP-16585:
Thanks [~surendrasingh], for reviewing the issue.
I have updated the patch.
> [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating
> file LoadGenerator#write
> ---
>
> Key: HADOOP-16585
> URL: https://issues.apache.org/jira/browse/HADOOP-16585
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ranith Sardar
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HADOOP-16585.001.patch, HADOOP-16585.002.patch
>
>
> {code:java}
> // id would be same for multiple file, so it may occur file not found
> exception
> private void write() throws IOException {
> String dirName = dirs.get(r.nextInt(dirs.size()));
> Path file = new Path(dirName, hostname+id);
>
> fc.delete(file, true);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
[
https://issues.apache.org/jira/browse/HADOOP-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ranith Sardar updated HADOOP-16585:
---
Attachment: HADOOP-16585.002.patch
> [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating
> file LoadGenerator#write
> ---
>
> Key: HADOOP-16585
> URL: https://issues.apache.org/jira/browse/HADOOP-16585
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ranith Sardar
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HADOOP-16585.001.patch, HADOOP-16585.002.patch
>
>
> {code:java}
> // id would be same for multiple file, so it may occur file not found
> exception
> private void write() throws IOException {
> String dirName = dirs.get(r.nextInt(dirs.size()));
> Path file = new Path(dirName, hostname+id);
>
> fc.delete(file, true);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1704: YARN-9958. Remove the invalid lock in ContainerExecutor
hadoop-yetus commented on issue #1704: YARN-9958. Remove the invalid lock in ContainerExecutor URL: https://github.com/apache/hadoop/pull/1704#issuecomment-553263376 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1337 | trunk passed | | +1 | compile | 71 | trunk passed | | +1 | checkstyle | 30 | trunk passed | | +1 | mvnsite | 41 | trunk passed | | +1 | shadedclient | 901 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 29 | trunk passed | | 0 | spotbugs | 80 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 77 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 36 | the patch passed | | +1 | compile | 60 | the patch passed | | +1 | javac | 60 | the patch passed | | +1 | checkstyle | 22 | the patch passed | | +1 | mvnsite | 34 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 886 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 29 | the patch passed | | +1 | findbugs | 97 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 1367 | hadoop-yarn-server-nodemanager in the patch passed. | | +1 | asflicense | 40 | The patch does not generate ASF License warnings. | | | | 5229 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1704/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1704 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4df438ce2b05 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1704/2/testReport/ | | Max. process+thread count | 308 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1704/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16455) ABFS: Implement FileSystem.access() method
[ https://issues.apache.org/jira/browse/HADOOP-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973041#comment-16973041 ] Bilahari T H edited comment on HADOOP-16455 at 11/13/19 6:15 AM: - Hi [~virajith], PR is up for this change. was (Author: bilahari.th): Hi [~virajith], PR is up for the same. > ABFS: Implement FileSystem.access() method > -- > > Key: HADOOP-16455 > URL: https://issues.apache.org/jira/browse/HADOOP-16455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > > Implement the access method -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16455) ABFS: Implement FileSystem.access() method
[ https://issues.apache.org/jira/browse/HADOOP-16455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973041#comment-16973041 ] Bilahari T H commented on HADOOP-16455: --- Hi [~virajith], PR is up for the same. > ABFS: Implement FileSystem.access() method > -- > > Key: HADOOP-16455 > URL: https://issues.apache.org/jira/browse/HADOOP-16455 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Bilahari T H >Assignee: Bilahari T H >Priority: Minor > > Implement the access method -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16699) ABFS: Enhance driver debug logs
[ https://issues.apache.org/jira/browse/HADOOP-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan updated HADOOP-16699: --- Status: Patch Available (was: Open) > ABFS: Enhance driver debug logs > > > Key: HADOOP-16699 > URL: https://issues.apache.org/jira/browse/HADOOP-16699 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Major > Fix For: 3.3.0 > > > Recently a hang observed in ABFS driver init code turned out to be an issue > in the client custom token provider. Before isolating the issue to custom > token provider, many suspects were listed such as hang in SSL context init or > UserGroup Init due to any environment or dependency factors and the driver > logs were unable to help conclude if it could be the case. > This Jira aims to be able to add enhanced verbose logging through TRACE level > which should help in investigations due to thread hang. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16699) ABFS: Enhance driver debug logs
[
https://issues.apache.org/jira/browse/HADOOP-16699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973033#comment-16973033
]
Sneha Vijayarajan commented on HADOOP-16699:
Test Results: From East US accounts.
Namespace not enabled:
{code:java}
[INFO] Results:
[INFO]
[INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[WARNING] Tests run: 396, Failures: 0, Errors: 0, Skipped: 211
[INFO]
[WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 126{code}
Namespace enabled:
{code:java}
[INFO] Results:
[INFO]
[INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Errors:
[ERROR] ITestAzureBlobFileSystemCLI>AbstractAbfsIntegrationTest.setup:137 »
AbfsRestOperation
[INFO]
[ERROR] Tests run: 396, Failures: 0, Errors: 1, Skipped: 25
[WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 126
{code}
The failing test needs to be rewritten as it has not taken into consideration
namespace enabled account scenario. Jira raised to handle this:
https://issues.apache.org/jira/browse/HADOOP-16687
> ABFS: Enhance driver debug logs
>
>
> Key: HADOOP-16699
> URL: https://issues.apache.org/jira/browse/HADOOP-16699
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Reporter: Sneha Vijayarajan
>Assignee: Sneha Vijayarajan
>Priority: Major
> Fix For: 3.3.0
>
>
> Recently a hang observed in ABFS driver init code turned out to be an issue
> in the client custom token provider. Before isolating the issue to custom
> token provider, many suspects were listed such as hang in SSL context init or
> UserGroup Init due to any environment or dependency factors and the driver
> logs were unable to help conclude if it could be the case.
> This Jira aims to be able to add enhanced verbose logging through TRACE level
> which should help in investigations due to thread hang.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1711: Hadoop 16455
hadoop-yetus commented on issue #1711: Hadoop 16455 URL: https://github.com/apache/hadoop/pull/1711#issuecomment-553241843 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1777 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1087 | trunk passed | | +1 | compile | 31 | trunk passed | | +1 | checkstyle | 26 | trunk passed | | +1 | mvnsite | 36 | trunk passed | | +1 | shadedclient | 808 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 52 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 51 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 28 | the patch passed | | +1 | compile | 24 | the patch passed | | +1 | javac | 24 | the patch passed | | -0 | checkstyle | 18 | hadoop-tools/hadoop-azure: The patch generated 3 new + 6 unchanged - 1 fixed = 9 total (was 7) | | +1 | mvnsite | 28 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 805 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | the patch passed | | +1 | findbugs | 54 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 82 | hadoop-azure in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 5047 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1711 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 3d8a165b0545 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-azure.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/testReport/ | | Max. process+thread count | 453 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1711/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp
hadoop-yetus commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp URL: https://github.com/apache/hadoop/pull/1669#issuecomment-553239059 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 71 | Maven dependency ordering for branch | | +1 | mvninstall | 1093 | trunk passed | | +1 | compile | 1012 | trunk passed | | +1 | checkstyle | 162 | trunk passed | | +1 | mvnsite | 169 | trunk passed | | +1 | shadedclient | 1141 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 186 | trunk passed | | 0 | spotbugs | 177 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 300 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | +1 | mvninstall | 115 | the patch passed | | +1 | compile | 974 | the patch passed | | +1 | javac | 974 | the patch passed | | +1 | checkstyle | 163 | the patch passed | | +1 | mvnsite | 167 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 1 | The patch has no ill-formed XML file. | | +1 | shadedclient | 762 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 194 | the patch passed | | +1 | findbugs | 320 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 554 | hadoop-common in the patch passed. | | -1 | unit | 5674 | hadoop-hdfs in the patch failed. | | +1 | asflicense | 60 | The patch does not generate ASF License warnings. | | | | 13169 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks | | | hadoop.hdfs.server.namenode.TestRedudantBlocks | | | hadoop.hdfs.tools.TestDFSZKFailoverController | | | hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1669/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1669 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux fd83a5cfb434 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / df6b316 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1669/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1669/2/testReport/ | | Max. process+thread count | 3876 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1669/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16703) Backport HADOOP-16152 to branch-3.1
[
https://issues.apache.org/jira/browse/HADOOP-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972986#comment-16972986
]
Hadoop QA commented on HADOOP-16703:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
56s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.1 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 9m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
56s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
49s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
0s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
48s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-client-modules/hadoop-client-minicluster
hadoop-client-modules/hadoop-client-runtime {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} branch-3.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
30s{color} | {color:green} branch-3.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 0s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-client-modules/hadoop-client-runtime
hadoop-client-modules/hadoop-client-minicluster {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
47s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}104m 17s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
14s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hadoop-client-runtime in the patch passed. {color} |
|
[GitHub] [hadoop] bilaharith opened a new pull request #1711: Hadoop 16455
bilaharith opened a new pull request #1711: Hadoop 16455 URL: https://github.com/apache/hadoop/pull/1711 mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify **Driver test results using an account with namespace support in Central India:** [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0 [ERROR] Errors: [ERROR] ITestAzureBlobFileSystemCLI>AbstractAbfsIntegrationTest.setup:137 ? AbfsRestOperation [INFO] [ERROR] Tests run: 405, Failures: 0, Errors: 1, Skipped: 33 [WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 24 **Driver test results using an account without namespace support in Central India:** NSD [INFO] Tests run: 42, Failures: 0, Errors: 0, Skipped: 0 [WARNING] Tests run: 405, Failures: 0, Errors: 0, Skipped: 220 WARNING] Tests run: 192, Failures: 0, Errors: 0, Skipped: 24 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp
ferhui commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp URL: https://github.com/apache/hadoop/pull/1669#issuecomment-553191160 @jojochuang Thanks for review. Add commit to remove improper lines This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16703) Backport HADOOP-16152 to branch-3.1
[ https://issues.apache.org/jira/browse/HADOOP-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HADOOP-16703: Attachment: HADOOP-16703.branch-3.1.001.patch Status: Patch Available (was: In Progress) > Backport HADOOP-16152 to branch-3.1 > --- > > Key: HADOOP-16703 > URL: https://issues.apache.org/jira/browse/HADOOP-16703 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16703.branch-3.1.001.patch > > > HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16703) Backport HADOOP-16152 to branch-3.1
[
https://issues.apache.org/jira/browse/HADOOP-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972869#comment-16972869
]
Siyao Meng commented on HADOOP-16703:
-
Uploading HADOOP-16703.branch-3.1.001.patch. I cherry-picked from branch-3.2
and manually resolved conflicts in:
{code}
hadoop-client-modules/hadoop-client-minicluster/pom.xml
hadoop-client-modules/hadoop-client-runtime/pom.xml
hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
{code}
Among them, SLSRunner doesn't need any change for branch-3.1 since it doesn't
use {{jetty.util.ConcurrentHashSet}}.
1. Compiled passed (with shading) locally
{code:bash}
mvn clean install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true
{code}
2. Ran shadedclient test and passed locally
{code:bash}
mvn --batch-mode verify -fae --batch-mode -am -pl
hadoop-client-modules/hadoop-client-check-invariants -pl
hadoop-client-modules/hadoop-client-check-test-invariants -pl
hadoop-client-modules/hadoop-client-integration-tests -Dtest=NoUnitTests
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true
{code}
> Backport HADOOP-16152 to branch-3.1
> ---
>
> Key: HADOOP-16703
> URL: https://issues.apache.org/jira/browse/HADOOP-16703
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.1.3
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
>
> HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp
jojochuang commented on issue #1669: HDFS-14802. The feature of protect directories should be used in RenameOp URL: https://github.com/apache/hadoop/pull/1669#issuecomment-553156207 @ferhui could you remove the line > [The feature of protected directories](#ProtectedDirectories) can be supported. Other than that I am +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15686) Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr
[
https://issues.apache.org/jira/browse/HADOOP-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972862#comment-16972862
]
Xiaoyu Yao commented on HADOOP-15686:
-
Thanks [~weichiu] for the update. The latest patch LGTM, I verified TestKMS
does not output bogus log messages after the latest patch.
Can you add some verification in TestKMS to capture the LOG output and make
sure it does not contain bogus log message in the description. This way, if
someone break it accidentally in future, we will be able to catch it as part of
the regression test.
> Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr
> -
>
> Key: HADOOP-15686
> URL: https://issues.apache.org/jira/browse/HADOOP-15686
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.0.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-15686.001.patch, HADOOP-15686.002.patch,
> HADOOP-15686.003.patch
>
>
> After we switched underlying system of KMS from Tomcat to Jetty, we started
> to observe a lot of bogus messages like the follow [1]. It is harmless but
> very annoying. Let's suppress it in log4j configuration.
> [1]
> {quote}
> Aug 20, 2018 11:26:17 AM
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> buildModelAndSchemas
> SEVERE: Failed to generate the schema for the JAX-B elements
> com.sun.xml.bind.v2.runtime.IllegalAnnotationsException: 2 counts of
> IllegalAnnotationExceptions
> java.util.Map is an interface, and JAXB can't handle interfaces.
> this problem is related to the following location:
> at java.util.Map
> java.util.Map does not have a no-arg default constructor.
> this problem is related to the following location:
> at java.util.Map
> at
> com.sun.xml.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:106)
> at
> com.sun.xml.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:489)
> at
> com.sun.xml.bind.v2.runtime.JAXBContextImpl.(JAXBContextImpl.java:319)
> at
> com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1170)
> at
> com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:145)
> at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:247)
> at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:234)
> at javax.xml.bind.ContextFinder.find(ContextFinder.java:441)
> at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:641)
> at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:584)
> at
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.buildModelAndSchemas(WadlGeneratorJAXBGrammarGenerator.java:169)
> at
> com.sun.jersey.server.wadl.generators.AbstractWadlGeneratorGrammarGenerator.createExternalGrammar(AbstractWadlGeneratorGrammarGenerator.java:405)
> at com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:149)
> at
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:119)
> at
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:138)
> at
> com.sun.jersey.server.impl.wadl.WadlMethodFactory$WadlOptionsMethodDispatcher.dispatch(WadlMethodFactory.java:110)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
>
[jira] [Work started] (HADOOP-16703) Backport HADOOP-16152 to branch-3.1
[ https://issues.apache.org/jira/browse/HADOOP-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-16703 started by Siyao Meng. --- > Backport HADOOP-16152 to branch-3.1 > --- > > Key: HADOOP-16703 > URL: https://issues.apache.org/jira/browse/HADOOP-16703 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Moved] (HADOOP-16703) Backport HADOOP-16152 to branch-3.1
[ https://issues.apache.org/jira/browse/HADOOP-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng moved HDFS-14982 to HADOOP-16703: Key: HADOOP-16703 (was: HDFS-14982) Target Version/s: 3.1.4 (was: 3.1.4) Affects Version/s: (was: 3.1.3) 3.1.3 Project: Hadoop Common (was: Hadoop HDFS) > Backport HADOOP-16152 to branch-3.1 > --- > > Key: HADOOP-16703 > URL: https://issues.apache.org/jira/browse/HADOOP-16703 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16677) Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
[
https://issues.apache.org/jira/browse/HADOOP-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972853#comment-16972853
]
Hudson commented on HADOOP-16677:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17637 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17637/])
HADOOP-16677. Recalculate the remaining timeout millis correctly while
(weichiu: rev df6b3162c11987ba5299c69cb251332228dacf36)
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/net/TestSocketIOWithTimeout.java
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java
> Recalculate the remaining timeout millis correctly while throwing an
> InterupptedException in SocketIOWithTimeout.
> -
>
> Key: HADOOP-16677
> URL: https://issues.apache.org/jira/browse/HADOOP-16677
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.1.3
>Reporter: Xudong Cao
>Assignee: Xudong Cao
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HADOOP-16677.001.patch
>
>
> In SocketIOWithTimeout, when a thread was interrupted and exit from select(),
> it proceed to throw an InterruptedIOException, in exception message the
> remaining timeout mills should be calcuated correctly rather than simply give
> a total timeout millis , which could be very misleading.
>
> An example log before this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. 6 millis timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
>
> An example log after this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. Total timeout mills is 6, 1000 millis
> timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16677) Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
[
https://issues.apache.org/jira/browse/HADOOP-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-16677:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
Done. Thanks for contributing the patch!
> Recalculate the remaining timeout millis correctly while throwing an
> InterupptedException in SocketIOWithTimeout.
> -
>
> Key: HADOOP-16677
> URL: https://issues.apache.org/jira/browse/HADOOP-16677
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.1.3
>Reporter: Xudong Cao
>Assignee: Xudong Cao
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HADOOP-16677.001.patch
>
>
> In SocketIOWithTimeout, when a thread was interrupted and exit from select(),
> it proceed to throw an InterruptedIOException, in exception message the
> remaining timeout mills should be calcuated correctly rather than simply give
> a total timeout millis , which could be very misleading.
>
> An example log before this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. 6 millis timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
>
> An example log after this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. Total timeout mills is 6, 1000 millis
> timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16677) Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
[
https://issues.apache.org/jira/browse/HADOOP-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-16677:
-
Fix Version/s: 3.2.2
3.1.4
> Recalculate the remaining timeout millis correctly while throwing an
> InterupptedException in SocketIOWithTimeout.
> -
>
> Key: HADOOP-16677
> URL: https://issues.apache.org/jira/browse/HADOOP-16677
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.1.3
>Reporter: Xudong Cao
>Assignee: Xudong Cao
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.3.0, 3.1.4, 3.2.2
>
> Attachments: HADOOP-16677.001.patch
>
>
> In SocketIOWithTimeout, when a thread was interrupted and exit from select(),
> it proceed to throw an InterruptedIOException, in exception message the
> remaining timeout mills should be calcuated correctly rather than simply give
> a total timeout millis , which could be very misleading.
>
> An example log before this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. 6 millis timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
>
> An example log after this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. Total timeout mills is 6, 1000 millis
> timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] smengcl edited a comment on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x
smengcl edited a comment on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x URL: https://github.com/apache/hadoop/pull/763#issuecomment-553143715 @aajisaka Hmm. Okay. I was just trying to run the HDFS tests to see which are broken by the upgrade. How am I supposed to check which ones are broken now that it won't compile? I see HDFS component is built. Somehow skip the YARN build and run the HDFS tests directly? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] smengcl commented on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x
smengcl commented on issue #763: [WIP] HADOOP-15984. Update jersey from 1.19 to 2.x URL: https://github.com/apache/hadoop/pull/763#issuecomment-553143715 @aajisaka Hmm. Okay. I was just trying to run the HDFS tests to see which ones are broken by the upgrade. How am I supposed to check which ones are broken? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang merged pull request #1687: HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
jojochuang merged pull request #1687: HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. URL: https://github.com/apache/hadoop/pull/1687 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on issue #1687: HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
jojochuang commented on issue #1687: HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. URL: https://github.com/apache/hadoop/pull/1687#issuecomment-553143280 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16677) Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout.
[
https://issues.apache.org/jira/browse/HADOOP-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-16677:
-
Fix Version/s: 3.3.0
> Recalculate the remaining timeout millis correctly while throwing an
> InterupptedException in SocketIOWithTimeout.
> -
>
> Key: HADOOP-16677
> URL: https://issues.apache.org/jira/browse/HADOOP-16677
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.1.3
>Reporter: Xudong Cao
>Assignee: Xudong Cao
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.3.0
>
> Attachments: HADOOP-16677.001.patch
>
>
> In SocketIOWithTimeout, when a thread was interrupted and exit from select(),
> it proceed to throw an InterruptedIOException, in exception message the
> remaining timeout mills should be calcuated correctly rather than simply give
> a total timeout millis , which could be very misleading.
>
> An example log before this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. 6 millis timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
>
> An example log after this jira:
> {code:java}
> 2019-10-31 16:20:39,496 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Exception for BP-1895911599-192.168.202.11-1572488735889:blk_1073741833_1016
> java.io.InterruptedIOException: Interrupted while waiting for IO on channel
> java.nio.channels.SocketChannel[connected local=/192.168.202.12:50010
> remote=/192.168.202.11:57006]. Total timeout mills is 6, 1000 millis
> timeout left.
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:351)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16676) Backport HADOOP-16152 to branch-3.2
[
https://issues.apache.org/jira/browse/HADOOP-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972773#comment-16972773
]
Siyao Meng commented on HADOOP-16676:
-
Sure thing. Will do. I believe there are just be some other minor conflicts in
{{pom.xml}}s.
> Backport HADOOP-16152 to branch-3.2
> ---
>
> Key: HADOOP-16676
> URL: https://issues.apache.org/jira/browse/HADOOP-16676
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.2.1
>Reporter: DW
>Assignee: Siyao Meng
>Priority: Major
> Fix For: 3.2.2
>
> Attachments: HADOOP-16676.branch-3.2.001.patch,
> HADOOP-16676.branch-3.2.001.patch, HADOOP-16676.branch-3.2.002.patch
>
>
> Hello,
>
> org.apache.hadoop:hadoop-common define the dependency to jetty-webapp and
> jetty-xml in version v9.3.24 with known CVE-2017-9735. Please can you upgrade
> to version 9.4.7 or higher?
> +--- org.apache.hadoop:hadoop-client:3.2.1
> | +--- org.apache.hadoop:hadoop-common:3.2.1
> | +--- org.eclipse.jetty:jetty-webapp:9.3.24.v20180605
> | | | +--- org.eclipse.jetty:jetty-xml:9.3.24.v20180605
> | | | \--- org.eclipse.jetty:jetty-servlet:9.3.24.v20180605 (*)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16656) Document FairCallQueue configs in core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972724#comment-16972724 ] Hudson commented on HADOOP-16656: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17635 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17635/]) HADOOP-16656. Document FairCallQueue configs in core-default.xml. (weichiu: rev 3ab98eefa54566e221b4b0f6416dd7065f5fd281) * (edit) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java > Document FairCallQueue configs in core-default.xml > -- > > Key: HADOOP-16656 > URL: https://issues.apache.org/jira/browse/HADOOP-16656 > Project: Hadoop Common > Issue Type: Task > Components: conf, documentation >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16656.001.patch, HADOOP-16656.002.patch, > HADOOP-16656.003.patch > > > So far those callqueue / scheduler / faircallqueue -related configurations > are only documented in FairCallQueue.md in 3.3.0: > https://aajisaka.github.io/hadoop-document/hadoop-project/hadoop-project-dist/hadoop-common/FairCallQueue.html#Full_List_of_Configurations > (Thanks Akira for uploading this.) > Goal: Document those configs in core-default.xml as well to make it easier > for users(admins) to find and use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16676) Backport HADOOP-16152 to branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972718#comment-16972718 ] Wei-Chiu Chuang commented on HADOOP-16676: -- Pushed to branch-3.2. The patch doesn't apply cleanly to branch-3.1. [~smeng] could you file another jira and provide a branch-3.1 patch? > Backport HADOOP-16152 to branch-3.2 > --- > > Key: HADOOP-16676 > URL: https://issues.apache.org/jira/browse/HADOOP-16676 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2.1 >Reporter: DW >Assignee: Siyao Meng >Priority: Major > Fix For: 3.2.2 > > Attachments: HADOOP-16676.branch-3.2.001.patch, > HADOOP-16676.branch-3.2.001.patch, HADOOP-16676.branch-3.2.002.patch > > > Hello, > > org.apache.hadoop:hadoop-common define the dependency to jetty-webapp and > jetty-xml in version v9.3.24 with known CVE-2017-9735. Please can you upgrade > to version 9.4.7 or higher? > +--- org.apache.hadoop:hadoop-client:3.2.1 > | +--- org.apache.hadoop:hadoop-common:3.2.1 > | +--- org.eclipse.jetty:jetty-webapp:9.3.24.v20180605 > | | | +--- org.eclipse.jetty:jetty-xml:9.3.24.v20180605 > | | | \--- org.eclipse.jetty:jetty-servlet:9.3.24.v20180605 (*) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16676) Backport HADOOP-16152 to branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16676: - Resolution: Fixed Status: Resolved (was: Patch Available) > Backport HADOOP-16152 to branch-3.2 > --- > > Key: HADOOP-16676 > URL: https://issues.apache.org/jira/browse/HADOOP-16676 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2.1 >Reporter: DW >Assignee: Siyao Meng >Priority: Major > Fix For: 3.2.2 > > Attachments: HADOOP-16676.branch-3.2.001.patch, > HADOOP-16676.branch-3.2.001.patch, HADOOP-16676.branch-3.2.002.patch > > > Hello, > > org.apache.hadoop:hadoop-common define the dependency to jetty-webapp and > jetty-xml in version v9.3.24 with known CVE-2017-9735. Please can you upgrade > to version 9.4.7 or higher? > +--- org.apache.hadoop:hadoop-client:3.2.1 > | +--- org.apache.hadoop:hadoop-common:3.2.1 > | +--- org.eclipse.jetty:jetty-webapp:9.3.24.v20180605 > | | | +--- org.eclipse.jetty:jetty-xml:9.3.24.v20180605 > | | | \--- org.eclipse.jetty:jetty-servlet:9.3.24.v20180605 (*) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16676) Backport HADOOP-16152 to branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16676: - Fix Version/s: 3.2.2 > Backport HADOOP-16152 to branch-3.2 > --- > > Key: HADOOP-16676 > URL: https://issues.apache.org/jira/browse/HADOOP-16676 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2.1 >Reporter: DW >Assignee: Siyao Meng >Priority: Major > Fix For: 3.2.2 > > Attachments: HADOOP-16676.branch-3.2.001.patch, > HADOOP-16676.branch-3.2.001.patch, HADOOP-16676.branch-3.2.002.patch > > > Hello, > > org.apache.hadoop:hadoop-common define the dependency to jetty-webapp and > jetty-xml in version v9.3.24 with known CVE-2017-9735. Please can you upgrade > to version 9.4.7 or higher? > +--- org.apache.hadoop:hadoop-client:3.2.1 > | +--- org.apache.hadoop:hadoop-common:3.2.1 > | +--- org.eclipse.jetty:jetty-webapp:9.3.24.v20180605 > | | | +--- org.eclipse.jetty:jetty-xml:9.3.24.v20180605 > | | | \--- org.eclipse.jetty:jetty-servlet:9.3.24.v20180605 (*) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16676) Backport HADOOP-16152 to branch-3.2
[ https://issues.apache.org/jira/browse/HADOOP-16676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16676: - Summary: Backport HADOOP-16152 to branch-3.2 (was: Security Vulnerability for dependency jetty-xml (Backport HADOOP-16152 to branch-3.2)) > Backport HADOOP-16152 to branch-3.2 > --- > > Key: HADOOP-16676 > URL: https://issues.apache.org/jira/browse/HADOOP-16676 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2.1 >Reporter: DW >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16676.branch-3.2.001.patch, > HADOOP-16676.branch-3.2.001.patch, HADOOP-16676.branch-3.2.002.patch > > > Hello, > > org.apache.hadoop:hadoop-common define the dependency to jetty-webapp and > jetty-xml in version v9.3.24 with known CVE-2017-9735. Please can you upgrade > to version 9.4.7 or higher? > +--- org.apache.hadoop:hadoop-client:3.2.1 > | +--- org.apache.hadoop:hadoop-common:3.2.1 > | +--- org.eclipse.jetty:jetty-webapp:9.3.24.v20180605 > | | | +--- org.eclipse.jetty:jetty-xml:9.3.24.v20180605 > | | | \--- org.eclipse.jetty:jetty-servlet:9.3.24.v20180605 (*) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16656) Document FairCallQueue configs in core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-16656: - Fix Version/s: 3.3.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed the change to trunk. Thanks! > Document FairCallQueue configs in core-default.xml > -- > > Key: HADOOP-16656 > URL: https://issues.apache.org/jira/browse/HADOOP-16656 > Project: Hadoop Common > Issue Type: Task > Components: conf, documentation >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16656.001.patch, HADOOP-16656.002.patch, > HADOOP-16656.003.patch > > > So far those callqueue / scheduler / faircallqueue -related configurations > are only documented in FairCallQueue.md in 3.3.0: > https://aajisaka.github.io/hadoop-document/hadoop-project/hadoop-project-dist/hadoop-common/FairCallQueue.html#Full_List_of_Configurations > (Thanks Akira for uploading this.) > Goal: Document those configs in core-default.xml as well to make it easier > for users(admins) to find and use. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang merged pull request #1706: HDFS-14959: [SBNN read] access time should be turned off
jojochuang merged pull request #1706: HDFS-14959: [SBNN read] access time should be turned off URL: https://github.com/apache/hadoop/pull/1706 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a change in pull request #1710: HADOOP-16692: UserGroupInformation Sets kerberosMinSecondsBeforeRelog…
goiri commented on a change in pull request #1710: HADOOP-16692:
UserGroupInformation Sets kerberosMinSecondsBeforeRelog…
URL: https://github.com/apache/hadoop/pull/1710#discussion_r345388302
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
##
@@ -336,7 +336,7 @@ private static synchronized void initialize(Configuration
conf,
}
}
try {
-kerberosMinSecondsBeforeRelogin = 1000L * conf.getLong(
+kerberosMinSecondsBeforeRelogin = conf.getLong(
Review comment:
Can we use getTimeDuration?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16612) Track Azure Blob File System client-perceived latency
[ https://issues.apache.org/jira/browse/HADOOP-16612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972690#comment-16972690 ] Íñigo Goiri commented on HADOOP-16612: -- +1 (I also approved the PR). In my opinion, we should enable checkstyle for this build. > Track Azure Blob File System client-perceived latency > - > > Key: HADOOP-16612 > URL: https://issues.apache.org/jira/browse/HADOOP-16612 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, hdfs-client >Reporter: Jeetesh Mangwani >Assignee: Jeetesh Mangwani >Priority: Major > Attachments: HADOOP-16612.001.patch, HADOOP-16612.002.patch, > HADOOP-16612.003.patch > > > Track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency > in the Hadoop ABFS driver. > The latency information is sent back to the ADLS Gen 2 REST API endpoints in > the subsequent requests. > Here's the PR: https://github.com/apache/hadoop/pull/1611 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16665) Filesystems to be closed if they failed during initialize()
[ https://issues.apache.org/jira/browse/HADOOP-16665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972683#comment-16972683 ] Hudson commented on HADOOP-16665: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17633 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17633/]) HADOOP-16665. Filesystems to be closed if they failed during (stevel: rev 990063d2af0a37e9474949f33128805e34c3f016) * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/HadoopTestBase.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ADataBlocks.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemInitialization.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/MockS3AFileSystem.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java * (add) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/AbstractHadoopTestBase.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestRestrictedReadAccess.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestAssumeRole.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/ITestAssumedRoleCommitOperations.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ABlockOutputStream.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java > Filesystems to be closed if they failed during initialize() > --- > > Key: HADOOP-16665 > URL: https://issues.apache.org/jira/browse/HADOOP-16665 > Project: Hadoop Common > Issue Type: Bug > Components: fs, fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Fix For: 3.3.0 > > > We seen this with S3A but it is likely to apply to the other file systems too: > If the file system raises an exception during initialize() then it is not > close()'d. Any resources threads etc created during initialisation may be > leaked. > I'm going to fix S3A to clean up internally, but the FileSystem get() and > newInstance() methods should also catch and close if an IOE or RTE is raised. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1705: HADOOP-16693: Review InterruptedException Handling
hadoop-yetus commented on issue #1705: HADOOP-16693: Review InterruptedException Handling URL: https://github.com/apache/hadoop/pull/1705#issuecomment-553038776 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1810 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1088 | trunk passed | | +1 | compile | 1024 | trunk passed | | +1 | checkstyle | 72 | trunk passed | | +1 | mvnsite | 84 | trunk passed | | +1 | shadedclient | 923 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 94 | trunk passed | | 0 | spotbugs | 131 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 128 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 48 | the patch passed | | +1 | compile | 966 | the patch passed | | -1 | javac | 966 | root generated 7 new + 1889 unchanged - 0 fixed = 1896 total (was 1889) | | -0 | checkstyle | 72 | hadoop-common-project/hadoop-common: The patch generated 1 new + 884 unchanged - 5 fixed = 885 total (was 889) | | +1 | mvnsite | 82 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 743 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 93 | the patch passed | | +1 | findbugs | 131 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 549 | hadoop-common in the patch passed. | | +1 | asflicense | 54 | The patch does not generate ASF License warnings. | | | | 8045 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1705 | | JIRA Issue | HADOOP-16693 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e59a41ce8ea8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f6697aa | | Default Java | 1.8.0_222 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/testReport/ | | Max. process+thread count | 1597 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16693) Review InterruptedException Handling
[
https://issues.apache.org/jira/browse/HADOOP-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972680#comment-16972680
]
Hadoop QA commented on HADOOP-16693:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 30m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 23s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
11s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 16m 6s{color}
| {color:red} root generated 7 new + 1889 unchanged - 0 fixed = 1896 total (was
1889) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 12s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 884 unchanged - 5 fixed = 885 total (was 889) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 23s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
9s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
54s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 base:
https://builds.apache.org/job/hadoop-multibranch/job/PR-1705/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/1705 |
| JIRA Issue | HADOOP-16693 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux e59a41ce8ea8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / f6697aa |
| Default Java | 1.8.0_222 |
| javac |
[jira] [Resolved] (HADOOP-16665) Filesystems to be closed if they failed during initialize()
[ https://issues.apache.org/jira/browse/HADOOP-16665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16665. - Fix Version/s: 3.3.0 Resolution: Fixed > Filesystems to be closed if they failed during initialize() > --- > > Key: HADOOP-16665 > URL: https://issues.apache.org/jira/browse/HADOOP-16665 > Project: Hadoop Common > Issue Type: Bug > Components: fs, fs/s3 >Affects Versions: 3.2.1, 3.1.3 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Fix For: 3.3.0 > > > We seen this with S3A but it is likely to apply to the other file systems too: > If the file system raises an exception during initialize() then it is not > close()'d. Any resources threads etc created during initialisation may be > leaked. > I'm going to fix S3A to clean up internally, but the FileSystem get() and > newInstance() methods should also catch and close if an IOE or RTE is raised. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16692) UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis
[
https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972673#comment-16972673
]
Hadoop QA commented on HADOOP-16692:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 2s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
8s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 45s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 49s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
28s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 base:
https://builds.apache.org/job/hadoop-multibranch/job/PR-1710/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/1710 |
| JIRA Issue | HADOOP-16692 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0c559a46675e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / f6697aa |
| Default Java | 1.8.0_222 |
| checkstyle |
[GitHub] [hadoop] hadoop-yetus commented on issue #1710: HADOOP-16692: UserGroupInformation Sets kerberosMinSecondsBeforeRelog…
hadoop-yetus commented on issue #1710: HADOOP-16692: UserGroupInformation Sets kerberosMinSecondsBeforeRelog… URL: https://github.com/apache/hadoop/pull/1710#issuecomment-553020021 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 77 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1271 | trunk passed | | +1 | compile | 1085 | trunk passed | | +1 | checkstyle | 45 | trunk passed | | +1 | mvnsite | 80 | trunk passed | | +1 | shadedclient | 962 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 89 | trunk passed | | 0 | spotbugs | 128 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 126 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 49 | the patch passed | | +1 | compile | 1026 | the patch passed | | +1 | javac | 1026 | the patch passed | | -0 | checkstyle | 45 | hadoop-common-project/hadoop-common: The patch generated 1 new + 77 unchanged - 2 fixed = 78 total (was 79) | | +1 | mvnsite | 77 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 829 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 88 | the patch passed | | +1 | findbugs | 133 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 568 | hadoop-common in the patch passed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 6676 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1710/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1710 | | JIRA Issue | HADOOP-16692 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0c559a46675e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f6697aa | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1710/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1710/1/testReport/ | | Max. process+thread count | 1796 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1710/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16694) Use Objects requireNonNull Where Appropriate
[
https://issues.apache.org/jira/browse/HADOOP-16694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972603#comment-16972603
]
David Mollitor commented on HADOOP-16694:
-
[~ste...@apache.org] Are you still good with replacing the current crop of {{if
(o == null) throw new NullPointerException();}} ?
> Use Objects requireNonNull Where Appropriate
>
>
> Key: HADOOP-16694
> URL: https://issues.apache.org/jira/browse/HADOOP-16694
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.2
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HADOOP-16694.1.patch, HADOOP-16694.2.patch,
> HADOOP-16694.3.patch
>
>
> https://docs.oracle.com/javase/8/docs/api/java/util/Objects.html#requireNonNull-T-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set
hadoop-yetus commented on issue #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set URL: https://github.com/apache/hadoop/pull/1210#issuecomment-552969306 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1259 | trunk passed | | +1 | compile | 32 | trunk passed | | +1 | checkstyle | 25 | trunk passed | | +1 | mvnsite | 37 | trunk passed | | +1 | shadedclient | 886 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 25 | trunk passed | | 0 | spotbugs | 59 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 56 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 32 | the patch passed | | +1 | compile | 27 | the patch passed | | +1 | javac | 27 | hadoop-tools_hadoop-aws generated 0 new + 15 unchanged - 1 fixed = 15 total (was 16) | | -0 | checkstyle | 18 | hadoop-tools/hadoop-aws: The patch generated 3 new + 13 unchanged - 0 fixed = 16 total (was 13) | | +1 | mvnsite | 32 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 934 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 26 | the patch passed | | +1 | findbugs | 76 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 85 | hadoop-aws in the patch failed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 3744 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.fs.s3a.s3guard.TestNullMetadataStore | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1210 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e5e887d9efc9 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / fb512f5 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/testReport/ | | Max. process+thread count | 421 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1210/12/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on a change in pull request #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set
hadoop-yetus commented on a change in pull request #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set URL: https://github.com/apache/hadoop/pull/1210#discussion_r345306960 ## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/assumed_roles.md ## @@ -758,14 +774,51 @@ Make sure that all the read and write permissions are allowed for any bucket/pat to which data is being written to, and read permissions for all buckets read from. +### `AccessDeniedException` When working with KMS-encrypted data + If the bucket is using SSE-KMS to encrypt data: 1. The caller must have the `kms:Decrypt` permission to read the data. -1. The caller needs `kms:Decrypt` and `kms:GenerateDataKey`. +1. The caller needs `kms:Decrypt` and `kms:GenerateDataKey` to write data. Without permissions, the request fails *and there is no explicit message indicating that this is an encryption-key issue*. +This problem is most obvious when you fail when writing data in a "Writing Object" operation. + +If the client does have write access to the bucket, verify that the caller has +`kms:GenerateDataKey` permissions for the encryption key in use. + +``` +java.nio.file.AccessDeniedException: test/testDTFileSystemClient: Writing Object on test/testDTFileSystemClient: + com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Review comment: whitespace:end of line This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
steveloughran commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552968184 thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16477) S3A delegation token tests fail if fs.s3a.encryption.key set
[ https://issues.apache.org/jira/browse/HADOOP-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972548#comment-16972548 ] Hudson commented on HADOOP-16477: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17631 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17631/]) HADOOP-16477. S3A delegation token tests fail if fs.s3a.encryption.key (stevel: rev f6697aa82b69af54dca6ac6b0eddacd0a74ede49) * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEC.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEKMSDefaultKey.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AMiscOperations.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/WriteOperationHelper.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEKMSUserDefinedKey.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/RolePolicies.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/ITestPartialRenamesDeletes.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/RoleTestUtils.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/AbstractTestS3AEncryption.java * (edit) hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/assumed_roles.md * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/AbstractDelegationIT.java * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFileystem.java > S3A delegation token tests fail if fs.s3a.encryption.key set > > > Key: HADOOP-16477 > URL: https://issues.apache.org/jira/browse/HADOOP-16477 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Fix For: 3.3.0 > > > 1. if you set an s3a encryption key, the Session and Role DelegationToken > tests fail...the test setup needs to unset that key for config and bucket > 2. SSE-C tests don't seem to be unsetting all their bucket properties -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16477) S3A delegation token tests fail if fs.s3a.encryption.key set
[ https://issues.apache.org/jira/browse/HADOOP-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16477. - Fix Version/s: 3.3.0 Resolution: Fixed committed to trunk > S3A delegation token tests fail if fs.s3a.encryption.key set > > > Key: HADOOP-16477 > URL: https://issues.apache.org/jira/browse/HADOOP-16477 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Fix For: 3.3.0 > > > 1. if you set an s3a encryption key, the Session and Role DelegationToken > tests fail...the test setup needs to unset that key for config and bucket > 2. SSE-C tests don't seem to be unsetting all their bucket properties -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16692) UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972529#comment-16972529 ] David Mollitor commented on HADOOP-16692: - [~ste...@apache.org] PR Submitted. Thanks! > UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis > --- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] belugabehr opened a new pull request #1710: HADOOP-16692: UserGroupInformation Sets kerberosMinSecondsBeforeRelog…
belugabehr opened a new pull request #1710: HADOOP-16692: UserGroupInformation Sets kerberosMinSecondsBeforeRelog… URL: https://github.com/apache/hadoop/pull/1710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16692) UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis
[ https://issues.apache.org/jira/browse/HADOOP-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HADOOP-16692: Summary: UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis (was: UserGroupInformation Treats kerberosMinSecondsBeforeRelogin as Millis) > UserGroupInformation Sets kerberosMinSecondsBeforeRelogin as Millis > --- > > Key: HADOOP-16692 > URL: https://issues.apache.org/jira/browse/HADOOP-16692 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HADOOP-16692.1.patch, HADOOP-16692.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] belugabehr commented on issue #1705: HADOOP-16693: Review InterruptedException Handling
belugabehr commented on issue #1705: HADOOP-16693: Review InterruptedException Handling URL: https://github.com/apache/hadoop/pull/1705#issuecomment-552941034 @steveloughran Thank you for the review!! Changes pushed to PR. Please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode
steveloughran commented on issue #1707: HADOOP-16697. Tune/audit auth mode URL: https://github.com/apache/hadoop/pull/1707#issuecomment-552940517 tests need to unset auth mode dirs from per bucket settings, else ITestAuthoritativePath can fail ``` [ERROR] testMultiAuthPath(org.apache.hadoop.fs.s3a.ITestAuthoritativePath) Time elapsed: 4.132 s <<< FAILURE! java.lang.AssertionError: Listing was not supposed to include s3a://hwdev-steve-ireland-new/test/1573571607290/testMultiAuthPath-first/in-path-out-of-band. Actual: s3a://hwdev-steve-ireland-new/test/1573571607290/testMultiAuthPath-first/in-path-out-of-band at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failEquals(Assert.java:185) at org.junit.Assert.assertNotEquals(Assert.java:161) at org.apache.hadoop.fs.s3a.S3ATestUtils.checkListingDoesNotContainPath(S3ATestUtils.java:1306) at org.apache.hadoop.fs.s3a.ITestAuthoritativePath.runTestInsidePath(ITestAuthoritativePath.java:213) at org.apache.hadoop.fs.s3a.ITestAuthoritativePath.testMultiAuthPath(ITestAuthoritativePath.java:251) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) [INFO] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
[
https://issues.apache.org/jira/browse/HADOOP-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972444#comment-16972444
]
Surendra Singh Lilhore commented on HADOOP-16585:
-
Thanks [~RANith] patch.
minor comment
Please change {{System.nanoTime()}} to \{{UUID.randomUUID().toString()}}.
> [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating
> file LoadGenerator#write
> ---
>
> Key: HADOOP-16585
> URL: https://issues.apache.org/jira/browse/HADOOP-16585
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ranith Sardar
>Assignee: Ranith Sardar
>Priority: Major
> Attachments: HADOOP-16585.001.patch
>
>
> {code:java}
> // id would be same for multiple file, so it may occur file not found
> exception
> private void write() throws IOException {
> String dirName = dirs.get(r.nextInt(dirs.size()));
> Path file = new Path(dirName, hostname+id);
>
> fc.delete(file, true);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1709: YARN-9966. Code duplication in UserGroupMappingPlacementRule
hadoop-yetus commented on issue #1709: YARN-9966. Code duplication in UserGroupMappingPlacementRule URL: https://github.com/apache/hadoop/pull/1709#issuecomment-552911677 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 2143 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1146 | trunk passed | | +1 | compile | 48 | trunk passed | | +1 | checkstyle | 39 | trunk passed | | +1 | mvnsite | 50 | trunk passed | | +1 | shadedclient | 842 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 35 | trunk passed | | 0 | spotbugs | 98 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 95 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 46 | the patch passed | | +1 | compile | 40 | the patch passed | | +1 | javac | 40 | the patch passed | | -0 | checkstyle | 31 | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 3 new + 11 unchanged - 4 fixed = 14 total (was 15) | | +1 | mvnsite | 45 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 797 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 32 | the patch passed | | +1 | findbugs | 99 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 5221 | hadoop-yarn-server-resourcemanager in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 10825 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1709/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1709 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux d991c9a81b62 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / fb512f5 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1709/1/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1709/1/testReport/ | | Max. process+thread count | 831 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1709/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bgaborg commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
bgaborg commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552877577 `ITestS3AContractRootDir` is not failing when I run it separately. I don't see how this is related to your work here, so it's a +1 from me on this because it will stabilize other things. +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bgaborg commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize().
bgaborg commented on issue #1671: HADOOP-16665. Filesystems to be closed if they failed during initialize(). URL: https://github.com/apache/hadoop/pull/1671#issuecomment-552876941 The same two tests are failing for me after the rerun. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15981) Add mutual TLS support for RPC
[ https://issues.apache.org/jira/browse/HADOOP-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972304#comment-16972304 ] Anu Engineer commented on HADOOP-15981: --- I just wanted to flag that Ozone has a complete Certificate Server and all the associated interface. We can bring it into Hadoop Common if there is some interest. it has an interface based approach, so that if we want to use an external CA, we can like the Key Management server or even have it running independently. > Add mutual TLS support for RPC > -- > > Key: HADOOP-15981 > URL: https://issues.apache.org/jira/browse/HADOOP-15981 > Project: Hadoop Common > Issue Type: Sub-task > Components: ipc, security >Reporter: Daryn Sharp >Assignee: Daryn Sharp >Priority: Major > > The RPC server should allow optionally enabling mutual TLS as 1st class > authentication. If enabled, a client cert may provide the user's identity or > fallback to kerberos or token. Essentially the placeholder CERTIFICATE > authentication method will be implemented and offered as an authentication > method during connection negotiation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set
steveloughran commented on issue #1210: HADOOP-16477. S3 delegation token tests fail if fs.s3a.encryption.key set URL: https://github.com/apache/hadoop/pull/1210#issuecomment-552845850 thanks for the vote Sid, will review the docs and retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pingsutw opened a new pull request #1709: YARN-9966. Code duplication in UserGroupMappingPlacementRule
pingsutw opened a new pull request #1709: YARN-9966. Code duplication in UserGroupMappingPlacementRule URL: https://github.com/apache/hadoop/pull/1709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bilaharith closed pull request #1213: Hadoop 16455
bilaharith closed pull request #1213: Hadoop 16455 URL: https://github.com/apache/hadoop/pull/1213 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16702) Move documentation of hadoop-cos to under src directory.
[ https://issues.apache.org/jira/browse/HADOOP-16702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972205#comment-16972205 ] Hudson commented on HADOOP-16702: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17629 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17629/]) HADOOP-16702. Move documentation of hadoop-cos to under src directory. (iwasakims: rev a98c4dfc47e0e4e20b38baf6e879a3e3da537ef2) * (delete) hadoop-cloud-storage-project/hadoop-cos/site/markdown/cloud-storage/index.md * (add) hadoop-cloud-storage-project/hadoop-cos/src/site/markdown/cloud-storage/index.md * (delete) hadoop-cloud-storage-project/hadoop-cos/site/resources/css/site.css * (add) hadoop-cloud-storage-project/hadoop-cos/src/site/resources/css/site.css * (edit) hadoop-project/src/site/site.xml > Move documentation of hadoop-cos to under src directory. > > > Key: HADOOP-16702 > URL: https://issues.apache.org/jira/browse/HADOOP-16702 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16702.001.patch > > > The documentation of hadoop-cos is not compiled because it is not under src > directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16702) Move documentation of hadoop-cos to under src directory.
[ https://issues.apache.org/jira/browse/HADOOP-16702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16702: -- Fix Version/s: 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Committed this. Thanks, [~aajisaka]! > Move documentation of hadoop-cos to under src directory. > > > Key: HADOOP-16702 > URL: https://issues.apache.org/jira/browse/HADOOP-16702 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Fix For: 3.3.0 > > Attachments: HADOOP-16702.001.patch > > > The documentation of hadoop-cos is not compiled because it is not under src > directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16702) Move documentation of hadoop-cos to under src directory.
[ https://issues.apache.org/jira/browse/HADOOP-16702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972168#comment-16972168 ] Akira Ajisaka commented on HADOOP-16702: +1 > Move documentation of hadoop-cos to under src directory. > > > Key: HADOOP-16702 > URL: https://issues.apache.org/jira/browse/HADOOP-16702 > Project: Hadoop Common > Issue Type: Bug > Components: tools >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Attachments: HADOOP-16702.001.patch > > > The documentation of hadoop-cos is not compiled because it is not under src > directory. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16701) Fix broken links in site index
[
https://issues.apache.org/jira/browse/HADOOP-16701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972163#comment-16972163
]
Hadoop QA commented on HADOOP-16701:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
32m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 57s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.4 Server=19.03.4 Image:yetus/hadoop:104ccca9169 |
| JIRA Issue | HADOOP-16701 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12985591/HADOOP-16701.003.patch
|
| Optional Tests | dupname asflicense mvnsite xml |
| uname | Linux c6719ec73784 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 30b93f9 |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 307 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16663/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Fix broken links in site index
> --
>
> Key: HADOOP-16701
> URL: https://issues.apache.org/jira/browse/HADOOP-16701
> Project: Hadoop Common
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.2.1
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HADOOP-16701.001.patch, HADOOP-16701.002.patch,
> HADOOP-16701.003.patch
>
>
> Fix links to removed/unexisting pages in site index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16701) Fix broken links in site index
[ https://issues.apache.org/jira/browse/HADOOP-16701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972160#comment-16972160 ] Akira Ajisaka commented on HADOOP-16701: Thank you, [~iwasakims]. > Fix broken links in site index > -- > > Key: HADOOP-16701 > URL: https://issues.apache.org/jira/browse/HADOOP-16701 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.2.1 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Fix For: 3.3.0 > > Attachments: HADOOP-16701.001.patch, HADOOP-16701.002.patch, > HADOOP-16701.003.patch > > > Fix links to removed/unexisting pages in site index. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16701) Fix broken links in site index
[ https://issues.apache.org/jira/browse/HADOOP-16701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972158#comment-16972158 ] Hudson commented on HADOOP-16701: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17628 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17628/]) HADOOP-16701. Fix broken links in site index. (iwasakims: rev b988487cae912247d48b6f9a2d65058695eaa124) * (edit) hadoop-project/src/site/site.xml > Fix broken links in site index > -- > > Key: HADOOP-16701 > URL: https://issues.apache.org/jira/browse/HADOOP-16701 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 3.2.1 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Fix For: 3.3.0 > > Attachments: HADOOP-16701.001.patch, HADOOP-16701.002.patch, > HADOOP-16701.003.patch > > > Fix links to removed/unexisting pages in site index. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
