[GitHub] [hadoop] dineshchitlangia commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project
dineshchitlangia commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project URL: https://github.com/apache/hadoop/pull/1340#issuecomment-532955927 No longer an issue and the version was bumped by another jira. Closing PR. Thanks all for reviewing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia closed pull request #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project
dineshchitlangia closed pull request #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project URL: https://github.com/apache/hadoop/pull/1340 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ChenSammi commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea…
ChenSammi commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea… URL: https://github.com/apache/hadoop/pull/1469#issuecomment-532945330 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #963: HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable
hadoop-yetus commented on issue #963: HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable URL: https://github.com/apache/hadoop/pull/963#issuecomment-532939526 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 42 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 5 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 69 | Maven dependency ordering for branch | | +1 | mvninstall | 1101 | trunk passed | | +1 | compile | 1172 | trunk passed | | +1 | checkstyle | 184 | trunk passed | | -1 | mvnsite | 160 | hadoop-common in trunk failed. | | +1 | shadedclient | 1396 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 214 | trunk passed | | 0 | spotbugs | 31 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 31 | branch/hadoop-hdfs-project/hadoop-hdfs-native-client no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | +1 | mvninstall | 194 | the patch passed | | +1 | compile | 1184 | the patch passed | | +1 | cc | 1184 | the patch passed | | +1 | javac | 1184 | the patch passed | | +1 | checkstyle | 221 | root: The patch generated 0 new + 110 unchanged - 1 fixed = 110 total (was 111) | | +1 | mvnsite | 284 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 828 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 214 | the patch passed | | 0 | findbugs | 28 | hadoop-hdfs-project/hadoop-hdfs-native-client has no data from findbugs | ||| _ Other Tests _ | | -1 | unit | 593 | hadoop-common in the patch failed. | | +1 | unit | 144 | hadoop-hdfs-client in the patch passed. | | -1 | unit | 5881 | hadoop-hdfs in the patch failed. | | +1 | unit | 407 | hadoop-hdfs-native-client in the patch passed. | | +1 | asflicense | 59 | The patch does not generate ASF License warnings. | | | | 15453 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ha.TestZKFailoverController | | | hadoop.hdfs.TestReadStripedFileWithDNFailure | | | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy | | | hadoop.hdfs.TestHdfsAdmin | | | hadoop.hdfs.TestErasureCodingPolicies | | | hadoop.hdfs.TestAppendSnapshotTruncate | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/963 | | JIRA Issue | HDFS-14564 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux fa1dfc47d245 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/artifact/out/branch-mvnsite-hadoop-common-project_hadoop-common.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/testReport/ | | Max. process+thread count | 4422 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-native-client U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-963/16/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail:
[GitHub] [hadoop] ChenSammi commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea…
ChenSammi commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea… URL: https://github.com/apache/hadoop/pull/1469#issuecomment-532936333 Majority of failed UT report "Could not initialize class org.apache.hadoop.ozone.util.OzoneVersionInfo". This class is not touched by the patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16543) Cached DNS name resolution error
[
https://issues.apache.org/jira/browse/HADOOP-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932989#comment-16932989
]
Hadoop QA commented on HADOOP-16543:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 32m
42s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
18s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
25s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue}
branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no findbugs output file
(findbugsXml.xml) {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 0m
59s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
15s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
22s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
32s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 32s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 28s{color} | {color:orange} The patch fails to run checkstyle in hadoop-yarn
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
19s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
23s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}
[GitHub] [hadoop] hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error
hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error URL: https://github.com/apache/hadoop/pull/1399#issuecomment-532933941 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1962 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 28 | Maven dependency ordering for branch | | +1 | mvninstall | 1241 | trunk passed | | +1 | compile | 529 | trunk passed | | +1 | checkstyle | 80 | trunk passed | | +1 | mvnsite | 159 | trunk passed | | +1 | shadedclient | 1000 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 138 | trunk passed | | 0 | spotbugs | 25 | Used deprecated FindBugs config; considering switching to SpotBugs. | | 0 | findbugs | 25 | branch/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site no findbugs output file (findbugsXml.xml) | | -0 | patch | 59 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for patch | | -1 | mvninstall | 15 | hadoop-yarn-common in the patch failed. | | -1 | mvninstall | 22 | hadoop-yarn-client in the patch failed. | | -1 | compile | 32 | hadoop-yarn in the patch failed. | | -1 | javac | 32 | hadoop-yarn in the patch failed. | | -0 | checkstyle | 28 | The patch fails to run checkstyle in hadoop-yarn | | -1 | mvnsite | 16 | hadoop-yarn-common in the patch failed. | | -1 | mvnsite | 24 | hadoop-yarn-client in the patch failed. | | -1 | whitespace | 0 | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 1 | The patch has no ill-formed XML file. | | -1 | shadedclient | 199 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 16 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 16 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 23 | hadoop-yarn-client in the patch failed. | | 0 | findbugs | 13 | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site has no data from findbugs | ||| _ Other Tests _ | | +1 | unit | 41 | hadoop-yarn-api in the patch passed. | | -1 | unit | 16 | hadoop-yarn-common in the patch failed. | | -1 | unit | 25 | hadoop-yarn-client in the patch failed. | | +1 | unit | 12 | hadoop-yarn-site in the patch passed. | | +1 | asflicense | 24 | The patch does not generate ASF License warnings. | | | | 6149 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1399 | | JIRA Issue | HADOOP-16543 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux cc71d73c597f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 4ed0aef | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1399@2/out/maven-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/7/artifact/out/whitespace-eol.txt | | javadoc |
[jira] [Commented] (HADOOP-15226) Über-JIRA: S3Guard Phase III: Hadoop 3.2 features
[ https://issues.apache.org/jira/browse/HADOOP-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932977#comment-16932977 ] Zhankun Tang commented on HADOOP-15226: --- Thanks for this valuable information. No. I don't think we need to pull it back. Thanks! [~steve_l] > Über-JIRA: S3Guard Phase III: Hadoop 3.2 features > - > > Key: HADOOP-15226 > URL: https://issues.apache.org/jira/browse/HADOOP-15226 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.0.0, 3.1.0 >Reporter: Steve Loughran >Priority: Major > Fix For: 3.2.0 > > > S3Guard features/improvements/fixes for Hadoop 3.2 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16543) Cached DNS name resolution error
[
https://issues.apache.org/jira/browse/HADOOP-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932955#comment-16932955
]
Hadoop QA commented on HADOOP-16543:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
56s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
55s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} trunk passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 1m
27s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
52s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 20s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 118 new + 214 unchanged - 0 fixed = 332 total (was 214) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 18s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
53s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
55s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 26m 20s{color}
| {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green}
[GitHub] [hadoop] hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error
hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error URL: https://github.com/apache/hadoop/pull/1399#issuecomment-532919790 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 35 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 1 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 43 | Maven dependency ordering for branch | | +1 | mvninstall | 1177 | trunk passed | | +1 | compile | 514 | trunk passed | | +1 | checkstyle | 79 | trunk passed | | +1 | mvnsite | 129 | trunk passed | | +1 | shadedclient | 959 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 116 | trunk passed | | 0 | spotbugs | 55 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 265 | trunk passed | | -0 | patch | 87 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 96 | the patch passed | | +1 | compile | 472 | the patch passed | | +1 | javac | 472 | the patch passed | | -0 | checkstyle | 80 | hadoop-yarn-project/hadoop-yarn: The patch generated 118 new + 214 unchanged - 0 fixed = 332 total (was 214) | | +1 | mvnsite | 122 | the patch passed | | -1 | whitespace | 0 | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 1 | The patch has no ill-formed XML file. | | +1 | shadedclient | 798 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 133 | the patch passed | | +1 | findbugs | 349 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 53 | hadoop-yarn-api in the patch passed. | | +1 | unit | 235 | hadoop-yarn-common in the patch passed. | | -1 | unit | 1580 | hadoop-yarn-client in the patch failed. | | +1 | asflicense | 41 | The patch does not generate ASF License warnings. | | | | 7306 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.yarn.client.api.impl.TestAMRMClient | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1399 | | JIRA Issue | HADOOP-16543 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 8bb2a14370c2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/testReport/ | | Max. process+thread count | 555 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16543) Cached DNS name resolution error
[
https://issues.apache.org/jira/browse/HADOOP-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932950#comment-16932950
]
Hadoop QA commented on HADOOP-16543:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
19s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
3s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} trunk passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 1m
43s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
35s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 25s{color} | {color:orange} The patch fails to run checkstyle in hadoop-yarn
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
26s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
58s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
20s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
45s{color} | {color:green}
[jira] [Commented] (HADOOP-16543) Cached DNS name resolution error
[
https://issues.apache.org/jira/browse/HADOOP-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932949#comment-16932949
]
Hadoop QA commented on HADOOP-16543:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 6m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
28m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
22s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
1s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} trunk passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 1m
36s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
34s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 34s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange} The patch fails to run checkstyle in hadoop-yarn
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
34s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green}
[GitHub] [hadoop] hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error
hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error URL: https://github.com/apache/hadoop/pull/1399#issuecomment-532918623 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 40 | Maven dependency ordering for branch | | +1 | mvninstall | 1031 | trunk passed | | +1 | compile | 497 | trunk passed | | +1 | checkstyle | 109 | trunk passed | | +1 | mvnsite | 157 | trunk passed | | +1 | shadedclient | 971 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 139 | trunk passed | | 0 | spotbugs | 63 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 274 | trunk passed | | -0 | patch | 103 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 18 | Maven dependency ordering for patch | | -1 | mvninstall | 17 | hadoop-yarn-common in the patch failed. | | -1 | mvninstall | 24 | hadoop-yarn-client in the patch failed. | | -1 | compile | 35 | hadoop-yarn in the patch failed. | | -1 | javac | 35 | hadoop-yarn in the patch failed. | | -0 | checkstyle | 25 | The patch fails to run checkstyle in hadoop-yarn | | -1 | mvnsite | 19 | hadoop-yarn-common in the patch failed. | | -1 | mvnsite | 26 | hadoop-yarn-client in the patch failed. | | -1 | whitespace | 0 | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 2 | The patch has no ill-formed XML file. | | -1 | shadedclient | 238 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 20 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 19 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 25 | hadoop-yarn-client in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 45 | hadoop-yarn-api in the patch passed. | | -1 | unit | 19 | hadoop-yarn-common in the patch failed. | | -1 | unit | 26 | hadoop-yarn-client in the patch failed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 3971 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1399 | | JIRA Issue | HADOOP-16543 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 9342eff06b9a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1399/out/maven-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/whitespace-eol.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/6/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | findbugs |
[GitHub] [hadoop] hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error
hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error URL: https://github.com/apache/hadoop/pull/1399#issuecomment-532918492 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 49 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 51 | Maven dependency ordering for branch | | +1 | mvninstall | 1146 | trunk passed | | +1 | compile | 1112 | trunk passed | | +1 | checkstyle | 194 | trunk passed | | +1 | mvnsite | 404 | trunk passed | | +1 | shadedclient | 1720 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 142 | trunk passed | | 0 | spotbugs | 61 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 280 | trunk passed | | -0 | patch | 96 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for patch | | -1 | mvninstall | 16 | hadoop-yarn-common in the patch failed. | | -1 | mvninstall | 25 | hadoop-yarn-client in the patch failed. | | -1 | compile | 34 | hadoop-yarn in the patch failed. | | -1 | javac | 34 | hadoop-yarn in the patch failed. | | -0 | checkstyle | 23 | The patch fails to run checkstyle in hadoop-yarn | | -1 | mvnsite | 19 | hadoop-yarn-common in the patch failed. | | -1 | mvnsite | 25 | hadoop-yarn-client in the patch failed. | | -1 | whitespace | 0 | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 2 | The patch has no ill-formed XML file. | | -1 | shadedclient | 214 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 19 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 18 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 25 | hadoop-yarn-client in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 44 | hadoop-yarn-api in the patch passed. | | -1 | unit | 18 | hadoop-yarn-common in the patch failed. | | -1 | unit | 25 | hadoop-yarn-client in the patch failed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 5460 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1399 | | JIRA Issue | HADOOP-16543 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 8afb5f5df49d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1399/out/maven-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/whitespace-eol.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/4/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | findbugs |
[GitHub] [hadoop] hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error
hadoop-yetus commented on issue #1399: HADOOP-16543: Cached DNS name resolution error URL: https://github.com/apache/hadoop/pull/1399#issuecomment-532915377 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 46 | Maven dependency ordering for branch | | +1 | mvninstall | 1189 | trunk passed | | +1 | compile | 538 | trunk passed | | +1 | checkstyle | 84 | trunk passed | | +1 | mvnsite | 154 | trunk passed | | +1 | shadedclient | 959 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 150 | trunk passed | | 0 | spotbugs | 64 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 286 | trunk passed | | -0 | patch | 102 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for patch | | -1 | mvninstall | 18 | hadoop-yarn-common in the patch failed. | | -1 | mvninstall | 24 | hadoop-yarn-client in the patch failed. | | -1 | compile | 38 | hadoop-yarn in the patch failed. | | -1 | javac | 38 | hadoop-yarn in the patch failed. | | -0 | checkstyle | 34 | The patch fails to run checkstyle in hadoop-yarn | | -1 | mvnsite | 19 | hadoop-yarn-common in the patch failed. | | -1 | mvnsite | 26 | hadoop-yarn-client in the patch failed. | | -1 | whitespace | 0 | The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 2 | The patch has no ill-formed XML file. | | -1 | shadedclient | 229 | patch has errors when building and testing our client artifacts. | | -1 | javadoc | 20 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 18 | hadoop-yarn-common in the patch failed. | | -1 | findbugs | 24 | hadoop-yarn-client in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 48 | hadoop-yarn-api in the patch passed. | | -1 | unit | 19 | hadoop-yarn-common in the patch failed. | | -1 | unit | 26 | hadoop-yarn-client in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 4278 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1399 | | JIRA Issue | HADOOP-16543 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml | | uname | Linux 7f6d6fb3b1d8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out//home/jenkins/jenkins-slave/workspace/hadoop-multibranch_PR-1399/out/maven-patch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | mvnsite | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/whitespace-eol.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1399/5/artifact/out/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt | | findbugs |
[jira] [Commented] (HADOOP-16543) Cached DNS name resolution error
[
https://issues.apache.org/jira/browse/HADOOP-16543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932941#comment-16932941
]
Hadoop QA commented on HADOOP-16543:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
4s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
46s{color} | {color:green} trunk passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 1m
42s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
38s{color} | {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 38s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 34s{color} | {color:orange} The patch fails to run checkstyle in hadoop-yarn
{color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
19s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
26s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 3m
49s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
20s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} hadoop-yarn-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} hadoop-yarn-client in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green}
[GitHub] [hadoop] hadoop-yetus commented on issue #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go…
hadoop-yetus commented on issue #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go… URL: https://github.com/apache/hadoop/pull/1463#issuecomment-532909740 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 42 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1057 | trunk passed | | +1 | compile | 1035 | trunk passed | | +1 | checkstyle | 51 | trunk passed | | +1 | mvnsite | 86 | trunk passed | | +1 | shadedclient | 833 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 73 | trunk passed | | 0 | spotbugs | 122 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 120 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 48 | the patch passed | | +1 | compile | 976 | the patch passed | | +1 | javac | 976 | the patch passed | | -0 | checkstyle | 47 | hadoop-common-project/hadoop-common: The patch generated 1 new + 24 unchanged - 0 fixed = 25 total (was 24) | | +1 | mvnsite | 83 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 784 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 71 | the patch passed | | +1 | findbugs | 140 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 597 | hadoop-common in the patch passed. | | +1 | asflicense | 50 | The patch does not generate ASF License warnings. | | | | 6157 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1463/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1463 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0ccbc799d30c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1463/3/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1463/3/testReport/ | | Max. process+thread count | 1377 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1463/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS name resolution error
RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS
name resolution error
URL: https://github.com/apache/hadoop/pull/1399#discussion_r325937828
##
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java
##
@@ -154,19 +151,44 @@ public T run() {
});
}
+
+ /**
+ * Helper method to create non-HA RMFailoverProxyProvider.
+ */
+ private RMFailoverProxyProvider createNonHaRMFailoverProxyProvider(
+ Configuration conf, Class protocol) {
+String defaultProviderClassName =
+YarnConfiguration.DEFAULT_CLIENT_FAILOVER_NO_HA_PROXY_PROVIDER;
+Class> defaultProviderClass;
+try {
+ defaultProviderClass = (Class>)
+ Class.forName(defaultProviderClassName);
+} catch (Exception e) {
+ throw new YarnRuntimeException("Invalid default failover provider class"
+
+ defaultProviderClassName, e);
+}
+
+RMFailoverProxyProvider provider = ReflectionUtils.newInstance(
+conf.getClass(YarnConfiguration.CLIENT_FAILOVER_NO_HA_PROXY_PROVIDER,
+defaultProviderClass, RMFailoverProxyProvider.class), conf);
+provider.init(conf, (RMProxy) this, protocol);
+return provider;
+ }
+
/**
* Helper method to create FailoverProxyProvider.
*/
private RMFailoverProxyProvider createRMFailoverProxyProvider(
Configuration conf, Class protocol) {
+String defaultProviderClassName =
Review comment:
My bad I thought this comment referred to a different issue.
I will change it back
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS name resolution error
RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS
name resolution error
URL: https://github.com/apache/hadoop/pull/1399#discussion_r325937426
##
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/RMProxy.java
##
@@ -125,16 +125,13 @@ public InetSocketAddress getRMAddress(
private static T newProxyInstance(final YarnConfiguration conf,
final Class protocol, RMProxy instance, RetryPolicy retryPolicy)
throws IOException{
+RMFailoverProxyProvider provider;
if (HAUtil.isHAEnabled(conf) || HAUtil.isFederationEnabled(conf)) {
- RMFailoverProxyProvider provider =
- instance.createRMFailoverProxyProvider(conf, protocol);
- return (T) RetryProxy.create(protocol, provider, retryPolicy);
+ provider = instance.createRMFailoverProxyProvider(conf, protocol);
} else {
- InetSocketAddress rmAddress = instance.getRMAddress(conf, protocol);
- LOG.info("Connecting to ResourceManager at " + rmAddress);
- T proxy = instance.getProxy(conf, protocol, rmAddress);
- return (T) RetryProxy.create(protocol, proxy, retryPolicy);
+ provider = instance.createNonHaRMFailoverProxyProvider(conf, protocol);
Review comment:
My previous comment was meant to refer to this piece of code sorry:
"We need this change so that the AutoRefreshNoHARMFailoverProxyProvider
works. Otherwise, it will try to get the IP address once and then fail if there
is ever a disconnect from the RM later"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao.
hadoop-yetus commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao. URL: https://github.com/apache/hadoop/pull/1369#issuecomment-532906357 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 79 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 14 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 70 | Maven dependency ordering for branch | | -1 | mvninstall | 29 | hadoop-ozone in trunk failed. | | -1 | compile | 19 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 82 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 922 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 48 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 173 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 24 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 24 | Maven dependency ordering for patch | | -1 | mvninstall | 30 | hadoop-ozone in the patch failed. | | -1 | compile | 22 | hadoop-ozone in the patch failed. | | -1 | cc | 22 | hadoop-ozone in the patch failed. | | -1 | javac | 22 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 41 | hadoop-hdds: The patch generated 69 new + 903 unchanged - 51 fixed = 972 total (was 954) | | -0 | checkstyle | 42 | hadoop-ozone: The patch generated 69 new + 981 unchanged - 56 fixed = 1050 total (was 1037) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 735 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 46 | hadoop-ozone in the patch failed. | | -1 | findbugs | 23 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 279 | hadoop-hdds in the patch failed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 3401 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdds.scm.container.placement.algorithms.TestSCMContainerPlacementRackAware | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1369 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml cc | | uname | Linux d4632650c2af 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-compile-hadoop-ozone.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/diff-checkstyle-hadoop-hdds.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/13/artifact/out/patch-unit-hadoop-hdds.txt | | unit |
[GitHub] [hadoop] hanishakoneru opened a new pull request #1472: HDDS-2150. Update dependency versions to avoid security vulnerabilities.
hanishakoneru opened a new pull request #1472: HDDS-2150. Update dependency versions to avoid security vulnerabilities. URL: https://github.com/apache/hadoop/pull/1472 The following dependency versions have known security vulnerabilities. We should update them to recent/ later versions. - Apache Thrift 0.11.0 (dependency of JaegerTracing) - Apache Zookeeper 3.4.13 - Jetty Servlet 9.3.24 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS name resolution error
RogPodge commented on a change in pull request #1399: HADOOP-16543: Cached DNS
name resolution error
URL: https://github.com/apache/hadoop/pull/1399#discussion_r325903846
##
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestNoHaRMFailoverProxyProvider.java
##
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.hadoop.yarn.client;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.io.retry.FailoverProxyProvider;
+import org.apache.hadoop.yarn.api.ApplicationClientProtocol;
+import org.apache.hadoop.yarn.api.records.NodeReport;
+import org.apache.hadoop.yarn.client.api.YarnClient;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+import org.apache.hadoop.yarn.server.MiniYARNCluster;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.lang.reflect.InvocationHandler;
+import java.lang.reflect.Proxy;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Random;
+
+import static org.mockito.Mockito.any;
+import static org.mockito.Mockito.eq;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+public class TestNoHaRMFailoverProxyProvider {
+private final int NODE_MANAGER_COUNT = 1;
+private Configuration conf;
+
+@Before
+public void setUp() throws IOException, YarnException {
+conf = new YarnConfiguration();
+}
+
+@Test
+public void testRestartedRM() throws Exception {
+MiniYARNCluster cluster =
+new MiniYARNCluster(
+"testRestartedRMNegative", NODE_MANAGER_COUNT, 1, 1);
+YarnClient rmClient = YarnClient.createYarnClient();
Review comment:
We need the rmClient to be stopped during the finally step so we need to
create it outside the try
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB
hadoop-yetus commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB URL: https://github.com/apache/hadoop/pull/1332#issuecomment-532875805 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 100 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1208 | trunk passed | | +1 | compile | 32 | trunk passed | | +1 | checkstyle | 23 | trunk passed | | +1 | mvnsite | 37 | trunk passed | | +1 | shadedclient | 806 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 24 | trunk passed | | 0 | spotbugs | 59 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 57 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 34 | the patch passed | | +1 | compile | 29 | the patch passed | | +1 | javac | 29 | the patch passed | | +1 | checkstyle | 21 | hadoop-tools/hadoop-aws: The patch generated 0 new + 42 unchanged - 1 fixed = 42 total (was 43) | | +1 | mvnsite | 35 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 875 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 23 | the patch passed | | +1 | findbugs | 64 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 85 | hadoop-aws in the patch passed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 3582 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1332 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 712a7a1f798a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_212 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/9/testReport/ | | Max. process+thread count | 310 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/9/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java
hadoop-yetus commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java URL: https://github.com/apache/hadoop/pull/1471#issuecomment-532873227 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 45 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 32 | hadoop-ozone in trunk failed. | | -1 | compile | 23 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 60 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 795 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 49 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 159 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 27 | hadoop-ozone in trunk failed. | | -0 | patch | 192 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 33 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 58 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 658 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in the patch failed. | | -1 | findbugs | 26 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 239 | hadoop-hdds in the patch passed. | | -1 | unit | 29 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 3080 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/Dockerfile | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux aa255146be8b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f16eb09 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/testReport/ | | Max. process+thread count | 482 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1471/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at:
[GitHub] [hadoop] arp7 commented on issue #1448: HDDS-2110. Arbitrary file can be downloaded with the help of ProfilerServlet
arp7 commented on issue #1448: HDDS-2110. Arbitrary file can be downloaded with the help of ProfilerServlet URL: https://github.com/apache/hadoop/pull/1448#issuecomment-532864393 Can this be committed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java
dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java URL: https://github.com/apache/hadoop/pull/1471#issuecomment-532863533 failures unrelated to patch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao.
hadoop-yetus commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao. URL: https://github.com/apache/hadoop/pull/1369#issuecomment-532860471 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 13 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 72 | Maven dependency ordering for branch | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | -1 | compile | 20 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 95 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 922 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 46 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 185 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 23 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | -1 | mvninstall | 29 | hadoop-ozone in the patch failed. | | -1 | compile | 21 | hadoop-ozone in the patch failed. | | -1 | cc | 21 | hadoop-ozone in the patch failed. | | -1 | javac | 21 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 38 | hadoop-hdds: The patch generated 63 new + 909 unchanged - 45 fixed = 972 total (was 954) | | -0 | checkstyle | 39 | hadoop-ozone: The patch generated 73 new + 977 unchanged - 60 fixed = 1050 total (was 1037) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 706 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 48 | hadoop-ozone in the patch failed. | | -1 | findbugs | 25 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 266 | hadoop-hdds in the patch passed. | | -1 | unit | 24 | hadoop-ozone in the patch failed. | | +1 | asflicense | 27 | The patch does not generate ASF License warnings. | | | | 3402 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1369 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml cc | | uname | Linux 2f82282865af 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 5db32b8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-compile-hadoop-ozone.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/diff-checkstyle-hadoop-hdds.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1369/11/testReport/ | | Max. process+thread count | 427 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/client hadoop-hdds/common
[GitHub] [hadoop] hadoop-yetus commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit.
hadoop-yetus commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit. URL: https://github.com/apache/hadoop/pull/1442#issuecomment-532857835 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 71 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1068 | trunk passed | | +1 | compile | 34 | trunk passed | | +1 | checkstyle | 26 | trunk passed | | +1 | mvnsite | 39 | trunk passed | | +1 | shadedclient | 732 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 60 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 58 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 35 | the patch passed | | +1 | compile | 27 | the patch passed | | +1 | javac | 27 | the patch passed | | -0 | checkstyle | 20 | hadoop-tools/hadoop-aws: The patch generated 2 new + 39 unchanged - 0 fixed = 41 total (was 39) | | +1 | mvnsite | 37 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 745 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 27 | the patch passed | | +1 | findbugs | 64 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 83 | hadoop-aws in the patch passed. | | +1 | asflicense | 35 | The patch does not generate ASF License warnings. | | | | 3228 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1442 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 310d4f645932 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 5db32b8 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/2/testReport/ | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea…
hadoop-yetus commented on issue #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea… URL: https://github.com/apache/hadoop/pull/1469#issuecomment-532857571 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 59 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 2 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 12 new or modified test files. | ||| _ HDDS-1564 Compile Tests _ | | 0 | mvndep | 31 | Maven dependency ordering for branch | | +1 | mvninstall | 732 | HDDS-1564 passed | | +1 | compile | 455 | HDDS-1564 passed | | +1 | checkstyle | 82 | HDDS-1564 passed | | +1 | mvnsite | 0 | HDDS-1564 passed | | +1 | shadedclient | 1059 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 199 | HDDS-1564 passed | | 0 | spotbugs | 505 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 739 | HDDS-1564 passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 39 | Maven dependency ordering for patch | | +1 | mvninstall | 678 | the patch passed | | +1 | compile | 444 | the patch passed | | +1 | cc | 444 | the patch passed | | +1 | javac | 444 | the patch passed | | -0 | checkstyle | 41 | hadoop-hdds: The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 848 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 88 | hadoop-hdds generated 2 new + 16 unchanged - 0 fixed = 18 total (was 16) | | -1 | findbugs | 254 | hadoop-hdds generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 322 | hadoop-hdds in the patch failed. | | -1 | unit | 1267 | hadoop-ozone in the patch failed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 8284 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-hdds | | | Null pointer dereference of dn in org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.onMessage(CommandStatusReportHandler$CreatePipelineStatus, EventPublisher) Dereferenced at SCMPipelineManager.java:in org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.onMessage(CommandStatusReportHandler$CreatePipelineStatus, EventPublisher) Dereferenced at SCMPipelineManager.java:[line 483] | | | Load of known null value in org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.onMessage(CommandStatusReportHandler$CreatePipelineStatus, EventPublisher) At SCMPipelineManager.java:in org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.onMessage(CommandStatusReportHandler$CreatePipelineStatus, EventPublisher) At SCMPipelineManager.java:[line 482] | | | The method name org.apache.hadoop.ozone.protocol.commands.CreatePipelineCommandStatus.SetCreatePipelineAck(StorageContainerDatanodeProtocolProtos$CreatePipelineACKProto) doesn't start with a lower case letter At CreatePipelineCommandStatus.java:start with a lower case letter At CreatePipelineCommandStatus.java:[lines 42-43] | | Failed junit tests | hadoop.hdds.scm.safemode.TestSCMSafeModeManager | | | hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerHandler | | | hadoop.ozone.container.common.impl.TestContainerDeletionChoosingPolicy | | | hadoop.ozone.TestMiniOzoneCluster | | | hadoop.ozone.dn.scrubber.TestDataScrubber | | | hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException | | | hadoop.ozone.om.TestOmMetrics | | | hadoop.hdds.scm.pipeline.TestPipelineStateManager | | | hadoop.ozone.client.rpc.TestContainerStateMachineFailures | | | hadoop.ozone.TestStorageContainerManager | | | hadoop.ozone.ozShell.TestOzoneDatanodeShell | | | hadoop.hdds.scm.pipeline.TestNode2PipelineMap | | | hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerByPipeline | | | hadoop.ozone.om.snapshot.TestOzoneManagerSnapshotProvider | | | hadoop.hdds.scm.pipeline.TestSCMPipelineManager | | | hadoop.hdds.scm.pipeline.TestPipelineClose | | | hadoop.ozone.web.client.TestKeysRatis | | | hadoop.ozone.om.TestOmAcls | | | hadoop.ozone.om.TestOzoneManagerRestart | | | hadoop.ozone.om.TestKeyPurging | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.om.TestScmSafeMode | | | hadoop.hdds.scm.pipeline.TestSCMRestart | | | hadoop.ozone.container.common.statemachine.commandhandler.TestDeleteContainerHandler | | |
[GitHub] [hadoop] bharatviswa504 commented on issue #1466: HDDS-2144. MR job failing on secure Ozone cluster.
bharatviswa504 commented on issue #1466: HDDS-2144. MR job failing on secure Ozone cluster. URL: https://github.com/apache/hadoop/pull/1466#issuecomment-532852312 Thank You @xiaoyuyao for the review. I have committed this to the trunk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 merged pull request #1466: HDDS-2144. MR job failing on secure Ozone cluster.
bharatviswa504 merged pull request #1466: HDDS-2144. MR job failing on secure Ozone cluster. URL: https://github.com/apache/hadoop/pull/1466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on issue #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go…
xkrogen commented on issue #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go… URL: https://github.com/apache/hadoop/pull/1463#issuecomment-532852247 LGTM, I pinged a few related folks on the JIRA. Also need to get Jenkins to run on this, I'm not sure why it isn't working. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar
bharatviswa504 commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar URL: https://github.com/apache/hadoop/pull/1452#issuecomment-532851470 `$ ./test.sh Removing network hadoop32_default WARNING: Network hadoop32_default not found. Creating network "hadoop32_default" with the default driver Creating hadoop32_scm_1 ... done Creating hadoop32_datanode_1 ... done Creating hadoop32_datanode_2 ... done Creating hadoop32_datanode_3 ... done Creating hadoop32_nm_1 ... done Creating hadoop32_rm_1 ... done Creating hadoop32_s3g_1 ... done Creating hadoop32_om_1 ... done 3 datanodes are up and registered to the scm == hadoop32-createmrenv :: Create directories required for MR test == Create test volume, bucket and key| PASS | -- Create user dir for hadoop| PASS | -- hadoop32-createmrenv :: Create directories required for MR test | PASS | 2 critical tests, 2 passed, 0 failed 2 tests total, 2 passed, 0 failed == Output: /tmp/smoketest/hadoop32/result/robot-hadoop32-hadoop32-createmrenv-scm.xml == hadoop32-hadoopo3fs :: Test ozone fs with hadoopfs == Test hadoop dfs | PASS | -- hadoop32-hadoopo3fs :: Test ozone fs with hadoopfs| PASS | 1 critical test, 1 passed, 0 failed 1 test total, 1 passed, 0 failed == Output: /tmp/smoketest/hadoop32/result/robot-hadoop32-hadoop32-hadoopo3fs-rm.xml == hadoop32-mapreduce :: Execute MR jobs == Execute PI calculation| PASS | -- Execute WordCount | PASS | -- hadoop32-mapreduce :: Execute MR jobs | PASS | 2 critical tests, 2 passed, 0 failed 2 tests total, 2 passed, 0 failed == Output: /tmp/smoketest/hadoop32/result/robot-hadoop32-hadoop32-mapreduce-rm.xml Stopping hadoop32_s3g_1 ... done Stopping hadoop32_datanode_1 ... done Stopping hadoop32_nm_1 ... done Stopping hadoop32_datanode_3 ... done Stopping hadoop32_datanode_2 ... done Stopping hadoop32_om_1 ... done Stopping hadoop32_scm_1 ... done Stopping hadoop32_rm_1 ... done Removing hadoop32_s3g_1 ... done Removing hadoop32_datanode_1 ... done Removing hadoop32_nm_1 ... done Removing hadoop32_datanode_3 ... done Removing hadoop32_datanode_2 ... done Removing hadoop32_om_1 ... done Removing hadoop32_scm_1 ... done Removing hadoop32_rm_1 ... done Removing network hadoop32_default Log: /Users/bviswanadham/workspace/myozone/hadoop/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozone-mr/hadoop32/result/log.html Report: /Users/bviswanadham/workspace/myozone/hadoop/hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozone-mr/hadoop32/result/report.html` Ran it locally test passed, the test failure is not related to this patch. I will commit this to the trunk. Thank You @elek for the improvement. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 merged pull request #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar
bharatviswa504 merged pull request #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar URL: https://github.com/apache/hadoop/pull/1452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] arp7 commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar
arp7 commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar URL: https://github.com/apache/hadoop/pull/1452#issuecomment-532840187 I assume all the test failures are unrelated. There is one SmokeTest failure for MapReduce. @bharatviswa504 ok to commit this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar
hadoop-yetus commented on issue #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar URL: https://github.com/apache/hadoop/pull/1452#issuecomment-532838197 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 2470 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 33 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-ozone in trunk failed. | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 981 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 50 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 33 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-ozone in the patch failed. | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 1 | The patch has no ill-formed XML file. | | +1 | shadedclient | 667 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 49 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 277 | hadoop-hdds in the patch failed. | | -1 | unit | 29 | hadoop-ozone in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 5053 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdds.scm.container.placement.algorithms.TestSCMContainerPlacementRackAware | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1452 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml | | uname | Linux fef328d63025 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 5db32b8 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/branch-javadoc-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-javadoc-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/testReport/ | | Max. process+thread count | 519 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozonefs-lib-current U: hadoop-ozone/ozonefs-lib-current | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1452/6/console | | versions | git=2.7.4 maven=3.3.9 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1470: HDDS-2147. Include dumpstream in test report
hadoop-yetus commented on issue #1470: HDDS-2147. Include dumpstream in test report URL: https://github.com/apache/hadoop/pull/1470#issuecomment-532836092 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 30 | hadoop-ozone in trunk failed. | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 765 | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 35 | hadoop-ozone in the patch failed. | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 2 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 659 | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 | unit | 58 | hadoop-hdds in the patch passed. | | -1 | unit | 31 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 1969 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1470 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs | | uname | Linux 17675525394c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 5db32b8 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/testReport/ | | Max. process+thread count | 412 (vs. ulimit of 5500) | | modules | C: hadoop-ozone U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1470/1/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16547) s3guard prune command doesn't get AWS auth chain from FS
[ https://issues.apache.org/jira/browse/HADOOP-16547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932766#comment-16932766 ] Hudson commented on HADOOP-16547: - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17326 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17326/]) HADOOP-16547. make sure that s3guard prune sets up the FS (#1402). (gabor.bota: rev 5db32b8ced8dc7533737caab88b97e151d2b223f) * (edit) hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/AbstractS3GuardToolTestBase.java * (edit) hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3GuardTool.java > s3guard prune command doesn't get AWS auth chain from FS > > > Key: HADOOP-16547 > URL: https://issues.apache.org/jira/browse/HADOOP-16547 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > s3guard prune command doesn't get AWS auth chain from any FS, so it just > drives the DDB store from the conf settings. If S3A is set up to use > Delegation tokens then the DTs/custom AWS auth sequence is not picked up, so > you get an auth failure. > Fix: > # instantiate the FS before calling initMetadataStore > # review other commands to make sure problem isn't replicated -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16570) S3A committers leak threads/raises OOM on job/task commit at scale
[ https://issues.apache.org/jira/browse/HADOOP-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16570: Summary: S3A committers leak threads/raises OOM on job/task commit at scale (was: S3A committers leak threads on job/task commit) > S3A committers leak threads/raises OOM on job/task commit at scale > -- > > Key: HADOOP-16570 > URL: https://issues.apache.org/jira/browse/HADOOP-16570 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.0, 3.1.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > The fixed size ThreadPool created in AbstractS3ACommitter doesn't get cleaned > up at EOL; as a result you leak the no. of threads set in > "fs.s3a.committer.threads" > Not visible in MR/distcp jobs, but ultimately causes OOM on Spark -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16570) S3A committers leak threads on job/task commit
[
https://issues.apache.org/jira/browse/HADOOP-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932742#comment-16932742
]
Steve Loughran commented on HADOOP-16570:
-
Even when you set the # of threads to 0, so disabling task commit, the job
commit (for a terasort BTW) fails.
stack trace implies its while listing the files to commit
{code}
main
at java.util.Arrays.copyOfRange([CII)[C (Arrays.java:3664)
at java.lang.String.([CII)V (String.java:207)
at java.lang.String.substring(II)Ljava/lang/String; (String.java:1969)
at java.net.URI$Parser.substring(II)Ljava/lang/String; (URI.java:2869)
at java.net.URI$Parser.parseHierarchical(II)I (URI.java:3106)
at java.net.URI$Parser.parse(Z)V (URI.java:3053)
at java.net.URI.(Ljava/lang/String;)V (URI.java:588)
at
org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit.destinationPath()Lorg/apache/hadoop/fs/Path;
(SinglePendingCommit.java:253)
at org.apache.hadoop.fs.s3a.commit.files.SinglePendingCommit.validate()V
(SinglePendingCommit.java:195)
at org.apache.hadoop.fs.s3a.commit.files.PendingSet.validate()V
(PendingSet.java:146)
at
org.apache.hadoop.fs.s3a.commit.files.PendingSet.load(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;)Lorg/apache/hadoop/fs/s3a/commit/files/PendingSet;
(PendingSet.java:109)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.lambda$loadPendingsetFiles$1(Ljava/util/List;Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/FileStatus;)V
(AbstractS3ACommitter.java:492)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter$$Lambda$92.run(Ljava/lang/Object;)V
(Unknown Source)
at
org.apache.hadoop.fs.s3a.commit.Tasks$Builder.runSingleThreaded(Lorg/apache/hadoop/fs/s3a/commit/Tasks$Task;)Z
(Tasks.java:165)
at
org.apache.hadoop.fs.s3a.commit.Tasks$Builder.run(Lorg/apache/hadoop/fs/s3a/commit/Tasks$Task;)Z
(Tasks.java:150)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.loadPendingsetFiles(Lorg/apache/hadoop/mapreduce/JobContext;ZLorg/apache/hadoop/fs/FileSystem;Ljava/lang/Iterable;)Ljava/util/List;
(AbstractS3ACommitter.java:490)
at
org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter.listPendingUploads(Lorg/apache/hadoop/mapreduce/JobContext;Z)Ljava/util/List;
(StagingCommitter.java:502)
at
org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter.listPendingUploadsToCommit(Lorg/apache/hadoop/mapreduce/JobContext;)Ljava/util/List;
(StagingCommitter.java:472)
at
org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter.commitJob(Lorg/apache/hadoop/mapreduce/JobContext;)V
(AbstractS3ACommitter.java:598)
at
org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(Lorg/apache/hadoop/mapreduce/JobContext;Lscala/collection/Seq;)V
(HadoopMapReduceCommitProtocol.scala:166)
at
org.apache.spark.internal.io.cloud.PathOutputCommitProtocol.commitJob(Lorg/apache/hadoop/mapreduce/JobContext;Lscala/collection/Seq;)V
(PathOutputCommitProtocol.scala:194)
at
{code}
Hypothesis: there are too many files to commit by way of enumerating them all
and then committing.
* We need to move to a sequence of load and commit or load and abort where both
the load and the commit/abort is done in the same worker thread.
* we don't create a list of results for a success file except for smaller jobs.
Maybe we could list the first 100 files and not worry about the rest; but do
add a counter of how many files there really were, if we didn't have one
already.
> S3A committers leak threads on job/task commit
> --
>
> Key: HADOOP-16570
> URL: https://issues.apache.org/jira/browse/HADOOP-16570
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.2.0, 3.1.2
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
>
> The fixed size ThreadPool created in AbstractS3ACommitter doesn't get cleaned
> up at EOL; as a result you leak the no. of threads set in
> "fs.s3a.committer.threads"
> Not visible in MR/distcp jobs, but ultimately causes OOM on Spark
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16582) LocalFileSystem's mkdirs() does not work as expected under viewfs.
[
https://issues.apache.org/jira/browse/HADOOP-16582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932734#comment-16932734
]
Hadoop QA commented on HADOOP-16582:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 145 unchanged - 3 fixed = 145 total (was 148) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 48s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
48s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:39e82acc485 |
| JIRA Issue | HADOOP-16582 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12980624/HADOOP-16582.1.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux db7fb26c97e4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 5dd859a |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16534/testReport/ |
| Max. process+thread count | 1421 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16534/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> LocalFileSystem's mkdirs() does not work as expected under viewfs.
[GitHub] [hadoop] sidseth commented on issue #1467: HADOOP-16583. Minor fixes to S3 testing instructions
sidseth commented on issue #1467: HADOOP-16583. Minor fixes to S3 testing instructions URL: https://github.com/apache/hadoop/pull/1467#issuecomment-532811794 Interesting. Now I'm confused. That command ran the tests, and skipped them when dynamo was skipped. Have been having a torrid time trying to get the S3 tests to work ... so lots of option switching on the command line, maybe something got mixed up there. I think this can be discarded. THere's a one line config change which is still valid. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java
dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java URL: https://github.com/apache/hadoop/pull/1471#issuecomment-532811572 @xiaoyuyao Pls help to review/commit this. TIA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java
dineshchitlangia commented on issue #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java URL: https://github.com/apache/hadoop/pull/1471#issuecomment-532810657 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16586) ITestS3GuardFsck, others fails when nun using a local metastore
[
https://issues.apache.org/jira/browse/HADOOP-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated HADOOP-16586:
Description:
Most of these tests fail if running against a local metastore with a
ClassCastException.
Not sure if these tests are intended to work with dynamo only. The fix (either
ignore in case of other metastores or fix the test) would depend on the
original intent.
{code}
---
Test set: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
---
Tests run: 12, Failures: 0, Errors: 11, Skipped: 1, Time elapsed: 34.653 s <<<
FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
testIDetectParentTombstoned(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.237 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentTombstoned(ITestS3GuardFsck.java:190)
testIDetectDirInS3FileInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 1.827 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectDirInS3FileInMs(ITestS3GuardFsck.java:214)
testIDetectLengthMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.819 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectLengthMismatch(ITestS3GuardFsck.java:311)
testIEtagMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time
elapsed: 2.832 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIEtagMismatch(ITestS3GuardFsck.java:373)
testIDetectFileInS3DirInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.752 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectFileInS3DirInMs(ITestS3GuardFsck.java:238)
testIDetectModTimeMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 4.103 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectModTimeMismatch(ITestS3GuardFsck.java:346)
testIDetectNoMetadataEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.017 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoMetadataEntry(ITestS3GuardFsck.java:113)
testIDetectNoParentEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.821 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoParentEntry(ITestS3GuardFsck.java:136)
testINoEtag(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time elapsed:
4.493 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testINoEtag(ITestS3GuardFsck.java:403)
testIDetectParentIsAFile(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.782 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentIsAFile(ITestS3GuardFsck.java:163)
testTombstonedInMsNotDeletedInS3(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.008 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
[GitHub] [hadoop] dineshchitlangia opened a new pull request #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java
dineshchitlangia opened a new pull request #1471: HDDS-2148. Remove redundant code in CreateBucketHandler.java URL: https://github.com/apache/hadoop/pull/1471 Removed redundant code, so no tests modified/added. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16586) ITestS3GuardFsck, others fails when nun using a local metastore
[
https://issues.apache.org/jira/browse/HADOOP-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated HADOOP-16586:
Description:
Most of these tests fail if running against a local metastore with a
ClassCastException.
Not sure if these tests are intended to work with dynamo only. The fix (either
ignore in case of other metastores or fix the test) would depend on the
original intent.
{code}
---
Test set: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
---
Tests run: 12, Failures: 0, Errors: 11, Skipped: 1, Time elapsed: 34.653 s <<<
FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
testIDetectParentTombstoned(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.237 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentTombstoned(ITestS3GuardFsck.java:190)
testIDetectDirInS3FileInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 1.827 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectDirInS3FileInMs(ITestS3GuardFsck.java:214)
testIDetectLengthMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.819 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectLengthMismatch(ITestS3GuardFsck.java:311)
testIEtagMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time
elapsed: 2.832 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIEtagMismatch(ITestS3GuardFsck.java:373)
testIDetectFileInS3DirInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.752 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectFileInS3DirInMs(ITestS3GuardFsck.java:238)
testIDetectModTimeMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 4.103 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectModTimeMismatch(ITestS3GuardFsck.java:346)
testIDetectNoMetadataEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.017 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoMetadataEntry(ITestS3GuardFsck.java:113)
testIDetectNoParentEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.821 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoParentEntry(ITestS3GuardFsck.java:136)
testINoEtag(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time elapsed:
4.493 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testINoEtag(ITestS3GuardFsck.java:403)
testIDetectParentIsAFile(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.782 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentIsAFile(ITestS3GuardFsck.java:163)
testTombstonedInMsNotDeletedInS3(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.008 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
[jira] [Updated] (HADOOP-16586) ITestS3GuardFsck, others fails when nun using a local metastore
[
https://issues.apache.org/jira/browse/HADOOP-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated HADOOP-16586:
Summary: ITestS3GuardFsck, others fails when nun using a local metastore
(was: ITestS3GuardFsck fails when nun using a local metastore)
> ITestS3GuardFsck, others fails when nun using a local metastore
> ---
>
> Key: HADOOP-16586
> URL: https://issues.apache.org/jira/browse/HADOOP-16586
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Reporter: Siddharth Seth
>Priority: Major
>
> Most of these tests fail if running against a local metastore with a
> ClassCastException.
> Not sure if these tests are intended to work with dynamo only. The fix
> (either ignore in case of other metastores or fix the test) would depend on
> the original intent.
> {code}
> ---
> Test set: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
> ---
> Tests run: 12, Failures: 0, Errors: 11, Skipped: 1, Time elapsed: 34.653 s
> <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
> testIDetectParentTombstoned(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 3.237 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentTombstoned(ITestS3GuardFsck.java:190)
> testIDetectDirInS3FileInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 1.827 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectDirInS3FileInMs(ITestS3GuardFsck.java:214)
> testIDetectLengthMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 2.819 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectLengthMismatch(ITestS3GuardFsck.java:311)
> testIEtagMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time
> elapsed: 2.832 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIEtagMismatch(ITestS3GuardFsck.java:373)
> testIDetectFileInS3DirInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 2.752 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectFileInS3DirInMs(ITestS3GuardFsck.java:238)
> testIDetectModTimeMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 4.103 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectModTimeMismatch(ITestS3GuardFsck.java:346)
> testIDetectNoMetadataEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 3.017 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoMetadataEntry(ITestS3GuardFsck.java:113)
> testIDetectNoParentEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time elapsed: 2.821 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoParentEntry(ITestS3GuardFsck.java:136)
> testINoEtag(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time elapsed:
> 4.493 s <<< ERROR!
> java.lang.ClassCastException:
> org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
> org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testINoEtag(ITestS3GuardFsck.java:403)
> testIDetectParentIsAFile(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
> Time
[jira] [Created] (HADOOP-16586) ITestS3GuardFsck fails when nun using a local metastore
Siddharth Seth created HADOOP-16586:
---
Summary: ITestS3GuardFsck fails when nun using a local metastore
Key: HADOOP-16586
URL: https://issues.apache.org/jira/browse/HADOOP-16586
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Reporter: Siddharth Seth
Most of these tests fail if running against a local metastore with a
ClassCastException.
Not sure if these tests are intended to work with dynamo only. The fix (either
ignore in case of other metastores or fix the test) would depend on the
original intent.
{code}
---
Test set: org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
---
Tests run: 12, Failures: 0, Errors: 11, Skipped: 1, Time elapsed: 34.653 s <<<
FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck
testIDetectParentTombstoned(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.237 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentTombstoned(ITestS3GuardFsck.java:190)
testIDetectDirInS3FileInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 1.827 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectDirInS3FileInMs(ITestS3GuardFsck.java:214)
testIDetectLengthMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.819 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectLengthMismatch(ITestS3GuardFsck.java:311)
testIEtagMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time
elapsed: 2.832 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIEtagMismatch(ITestS3GuardFsck.java:373)
testIDetectFileInS3DirInMs(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.752 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectFileInS3DirInMs(ITestS3GuardFsck.java:238)
testIDetectModTimeMismatch(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 4.103 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectModTimeMismatch(ITestS3GuardFsck.java:346)
testIDetectNoMetadataEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.017 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoMetadataEntry(ITestS3GuardFsck.java:113)
testIDetectNoParentEntry(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.821 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectNoParentEntry(ITestS3GuardFsck.java:136)
testINoEtag(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck) Time elapsed:
4.493 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testINoEtag(ITestS3GuardFsck.java:403)
testIDetectParentIsAFile(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 2.782 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore cannot be cast to
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore
at
org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck.testIDetectParentIsAFile(ITestS3GuardFsck.java:163)
testTombstonedInMsNotDeletedInS3(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardFsck)
Time elapsed: 3.008 s <<< ERROR!
java.lang.ClassCastException:
org.apache.hadoop.fs.s3a.s3guard.LocalMetadataStore
[GitHub] [hadoop] steveloughran commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit.
steveloughran commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit. URL: https://github.com/apache/hadoop/pull/1442#issuecomment-532793888 Note that the thread tests go beyond just asking the committers if they have a thread pool; they verify that there aren't any left around. If we like this, it could be expanded to other places, such as verifying the other FS clients are also being good citizens -especially for hive, which creates and destroys many instances This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit.
steveloughran commented on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit. URL: https://github.com/apache/hadoop/pull/1442#issuecomment-532790384 Tested, s3a ireland. FWIW, I'm planning to backport the thread cleanup and extra asserts to branches -3.1 and 3.2; I'm not so sure about the thread checks if we need assertj to be added, but well, its an extra test-time dependency so not that troublesome. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit.
hadoop-yetus removed a comment on issue #1442: HADOOP-16570. S3A committers leak threads on job/task commit. URL: https://github.com/apache/hadoop/pull/1442#issuecomment-531357838 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1142 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1198 | trunk passed | | +1 | compile | 34 | trunk passed | | +1 | checkstyle | 27 | trunk passed | | +1 | mvnsite | 36 | trunk passed | | +1 | shadedclient | 777 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 26 | trunk passed | | 0 | spotbugs | 60 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 56 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 33 | the patch passed | | +1 | compile | 27 | the patch passed | | +1 | javac | 27 | the patch passed | | +1 | checkstyle | 18 | the patch passed | | +1 | mvnsite | 30 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 816 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 23 | the patch passed | | +1 | findbugs | 61 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 74 | hadoop-aws in the patch passed. | | +1 | asflicense | 28 | The patch does not generate ASF License warnings. | | | | 4502 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1442 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux abef849a9c36 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 1843c46 | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/1/testReport/ | | Max. process+thread count | 353 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1442/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16547) s3guard prune command doesn't get AWS auth chain from FS
[ https://issues.apache.org/jira/browse/HADOOP-16547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota resolved HADOOP-16547. - Resolution: Fixed > s3guard prune command doesn't get AWS auth chain from FS > > > Key: HADOOP-16547 > URL: https://issues.apache.org/jira/browse/HADOOP-16547 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > s3guard prune command doesn't get AWS auth chain from any FS, so it just > drives the DDB store from the conf settings. If S3A is set up to use > Delegation tokens then the DTs/custom AWS auth sequence is not picked up, so > you get an auth failure. > Fix: > # instantiate the FS before calling initMetadataStore > # review other commands to make sure problem isn't replicated -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16547) s3guard prune command doesn't get AWS auth chain from FS
[ https://issues.apache.org/jira/browse/HADOOP-16547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932692#comment-16932692 ] Gabor Bota commented on HADOOP-16547: - +1 on PR 1402. Committed. > s3guard prune command doesn't get AWS auth chain from FS > > > Key: HADOOP-16547 > URL: https://issues.apache.org/jira/browse/HADOOP-16547 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > s3guard prune command doesn't get AWS auth chain from any FS, so it just > drives the DDB store from the conf settings. If S3A is set up to use > Delegation tokens then the DTs/custom AWS auth sequence is not picked up, so > you get an auth failure. > Fix: > # instantiate the FS before calling initMetadataStore > # review other commands to make sure problem isn't replicated -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bgaborg merged pull request #1402: HADOOP-16547. make sure that s3guard prune sets up the FS
bgaborg merged pull request #1402: HADOOP-16547. make sure that s3guard prune sets up the FS URL: https://github.com/apache/hadoop/pull/1402 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant merged pull request #1420: HDDS-2032. Ozone client should retry writes in case of any ratis/stateMachine exceptions.
bshashikant merged pull request #1420: HDDS-2032. Ozone client should retry writes in case of any ratis/stateMachine exceptions. URL: https://github.com/apache/hadoop/pull/1420 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1470: HDDS-2147. Include dumpstream in test report
adoroszlai commented on issue #1470: HDDS-2147. Include dumpstream in test report URL: https://github.com/apache/hadoop/pull/1470#issuecomment-532772320 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai opened a new pull request #1470: HDDS-2147. Include dumpstream in test report
adoroszlai opened a new pull request #1470: HDDS-2147. Include dumpstream in test report URL: https://github.com/apache/hadoop/pull/1470 ## What changes were proposed in this pull request? Copy `*.dumpstream` files to the unit test report dir. It may help finding out the cause of `Corrupted STDOUT` warning of forked JVM. http://maven.apache.org/surefire/maven-surefire-plugin/faq.html#dumpfiles https://issues.apache.org/jira/browse/HDDS-2147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16581) ValueQueue does not trigger an async refill when number of values falls below watermark
[ https://issues.apache.org/jira/browse/HADOOP-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932659#comment-16932659 ] Erik Krogen commented on HADOOP-16581: -- The change LGTM, ping [~xiaochen] who I see has some previous commits in this file, and [~anu] / [~daryn] / [~kihwal] who I believe have some experience with the KMS. > ValueQueue does not trigger an async refill when number of values falls below > watermark > --- > > Key: HADOOP-16581 > URL: https://issues.apache.org/jira/browse/HADOOP-16581 > Project: Hadoop Common > Issue Type: Bug > Components: common, kms >Affects Versions: 2.7.4, 3.2.0 >Reporter: Yuval Degani >Assignee: Yuval Degani >Priority: Major > Fix For: 3.2.1 > > > The ValueQueue facility was designed to cache EDEKs for KMS KeyProviders so > that EDEKs could be served quickly, while the cache is replenished in a > background thread. > The existing code for triggering an asynchronous refill is only triggered > when a key queue becomes empty, rather than when it falls below the > configured watermark. > This is a relatively minor fix in the main code, however, most of the tests > require some changes as they verify the previous unintended behavior. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go…
xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go… URL: https://github.com/apache/hadoop/pull/1463#discussion_r325778045 ## File path: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java ## @@ -22,6 +22,7 @@ import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; Review comment: minor nit, can you put this next to the other `java.util` imports? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go…
xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise
ValueQueue to correctly replenish queues that go…
URL: https://github.com/apache/hadoop/pull/1463#discussion_r325778180
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java
##
@@ -62,14 +63,29 @@ public FillInfo getTop() throws InterruptedException {
}
}
+ private void waitForRefill(ValueQueue valueQueue, String queueName, int
queueSize)
+ throws TimeoutException, InterruptedException {
+GenericTestUtils.waitFor(new Supplier() {
Review comment:
Consider using a lambda here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise ValueQueue to correctly replenish queues that go…
xkrogen commented on a change in pull request #1463: HADOOP-16581. Revise
ValueQueue to correctly replenish queues that go…
URL: https://github.com/apache/hadoop/pull/1463#discussion_r325779945
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/key/TestValueQueue.java
##
@@ -106,14 +122,15 @@ public void testWarmUp() throws Exception {
public void testRefill() throws Exception {
MockFiller filler = new MockFiller();
ValueQueue vq =
-new ValueQueue(10, 0.1f, 300, 1,
+new ValueQueue(100, 0.1f, 3, 1,
SyncGenerationPolicy.ALL, filler);
+// Trigger a prefill (10) and an async refill (91)
Assert.assertEquals("test", vq.getNext("k1"));
-Assert.assertEquals(1, filler.getTop().num);
-// Trigger refill
-vq.getNext("k1");
-Assert.assertEquals(1, filler.getTop().num);
Assert.assertEquals(10, filler.getTop().num);
+
+// Wait for the async task to finish
+waitForRefill(vq, "k1", 100);
+Assert.assertEquals(91, filler.getTop().num);
Review comment:
Can you explain in a comment where the number 91 comes from? If I understand
correctly it's because the async replication task has to fill 91 values (`= 100
- 9`)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao.
xiaoyuyao commented on issue #1369: HDDS-2020. Remove mTLS from Ozone GRPC. Contributed by Xiaoyu Yao. URL: https://github.com/apache/hadoop/pull/1369#issuecomment-532768160 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on issue #1466: HDDS-2144. MR job failing on secure Ozone cluster.
xiaoyuyao commented on issue #1466: HDDS-2144. MR job failing on secure Ozone cluster. URL: https://github.com/apache/hadoop/pull/1466#issuecomment-532767551 LGTM, +1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on a change in pull request #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar
elek commented on a change in pull request #1452: HDDS-2121. Create a shaded ozone filesystem (client) jar URL: https://github.com/apache/hadoop/pull/1452#discussion_r325769031 ## File path: hadoop-ozone/ozonefs-lib-current/pom.xml ## @@ -83,6 +63,78 @@ true + +org.apache.maven.plugins +maven-shade-plugin + + +package + + shade + + + + + classworlds:classworlds Review comment: No, we don't. Thanks to point to it. Removed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on issue #1387: HDDS-2065. Implement OMNodeDetails#toString
bharatviswa504 commented on issue #1387: HDDS-2065. Implement OMNodeDetails#toString URL: https://github.com/apache/hadoop/pull/1387#issuecomment-532746543 Thank You @elek for sharing info. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ChenSammi opened a new pull request #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea…
ChenSammi opened a new pull request #1469: HDDS-2034. Async RATIS pipeline creation and destroy through heartbea… URL: https://github.com/apache/hadoop/pull/1469 …t commands. https://issues.apache.org/jira/browse/HDDS-2034?filter=-1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1387: HDDS-2065. Implement OMNodeDetails#toString
elek closed pull request #1387: HDDS-2065. Implement OMNodeDetails#toString URL: https://github.com/apache/hadoop/pull/1387 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1465: HDDS-2143. Rename classes under package org.apache.hadoop.utils.
elek closed pull request #1465: HDDS-2143. Rename classes under package org.apache.hadoop.utils. URL: https://github.com/apache/hadoop/pull/1465 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16580) Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-16580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932522#comment-16932522
]
Adam Antal commented on HADOOP-16580:
-
Thanks for the review [~shuzirra], let me give you a bit more clarification on
the patch.
UnreliableInterface is an interface for *purely testing purpose*. It is used
for testing RetryPolicy applied to the UnreliableInterface which mimics an
interface that is "unreliable" in a sense that it might throws exceptions. It
is used for checking that the RetryPolicy is triggered / works as expected
having an unreliable underlying resource (UnreliableImplementation) throwing
Exception in particular circumstances.
The annotation is something that is needed for the imitation - the API calls
(like {{mapred job -list}} - see HADOOP-14982) are usually annotated like that,
and in the code there's a part where:
{code:java}
...
} else if (e instanceof SocketException
|| (e instanceof IOException && !(e instanceof RemoteException))) {
if (isIdempotentOrAtMostOnce) {
return new RetryAction(RetryAction.RetryDecision.FAILOVER_AND_RETRY,
getFailoverOrRetrySleepTime(retries));
} else {
return new RetryAction(RetryAction.RetryDecision.FAIL, 0,
"the invoked method is not idempotent, and unable to determine "
+ "whether it was invoked");
}
} ...
{code}
where the {{isIdempotentOrAtMostOnce}} variable is determined by the
interface's annotation. Through reflection it is queried whether the method has
the {{@AtMostOnce}} or {{@Idempotent}} annotation and the
{{FailoverOnNetworkExceptionRetry}} RetryPolicy decides accordingly to retry or
fail.
Since I wanted to test the RetryPolicy on an Idempotent-annotated call (just
like the API call) which throws an AccessControlException I think the test is
logically correct, and the annotation is proper for this reason.
As for the name of the method:
The counter starts from 0, and each time the function is called it is
incremented by one and will throw exception each time until it reaches 8 (which
is the 9th call) where it won't emit any new exception. I think this is the
exact behaviour that the name suggests. If there's any other issue or adding
some javadoc to make this clearer I can work on that. It was hard to understand
me these concepts first since the lack of javadoc, but hope the above
description helps.
> Disable retry of FailoverOnNetworkExceptionRetry in case of
> AccessControlException
> --
>
> Key: HADOOP-16580
> URL: https://issues.apache.org/jira/browse/HADOOP-16580
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Major
> Attachments: HADOOP-16580.001.patch, HADOOP-16580.002.patch
>
>
> HADOOP-14982 handled the case where a SaslException is thrown. The issue
> still persists, since the exception that is thrown is an
> *AccessControlException* because user has no kerberos credentials.
> My suggestion is that we should add this case as well to
> {{FailoverOnNetworkExceptionRetry}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1455: HDDS-2137 : OzoneUtils to verify resourceName using HddsClientUtils
elek closed pull request #1455: HDDS-2137 : OzoneUtils to verify resourceName using HddsClientUtils URL: https://github.com/apache/hadoop/pull/1455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1468: HDDS-2138. OM bucket operations do not add up
adoroszlai commented on issue #1468: HDDS-2138. OM bucket operations do not add up URL: https://github.com/apache/hadoop/pull/1468#issuecomment-532717453 Thanks @elek for reviewing and committing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16582) LocalFileSystem's mkdirs() does not work as expected under viewfs.
[
https://issues.apache.org/jira/browse/HADOOP-16582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932514#comment-16932514
]
Kihwal Lee commented on HADOOP-16582:
-
Also fixed the existing mkdirs() method, which had the same checkstyle issue.
> LocalFileSystem's mkdirs() does not work as expected under viewfs.
> --
>
> Key: HADOOP-16582
> URL: https://issues.apache.org/jira/browse/HADOOP-16582
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Major
> Attachments: HADOOP-16582.1.patch, HADOOP-16582.patch
>
>
> When {{mkdirs(Path)}} is called against {{LocalFileSystem}}, the
> implementation in {{RawLocalFileSystem}} is called and the directory
> permission is determined by the umask. However, if it is under
> {{ViewFileSystem}}, the default implementation in {{FileSystem}} is called
> and this causes explicit {{chmod()}} to 0777.
> The {{mkdirs(Path)}} method needs to be overriden in
> - ViewFileSystem to avoid calling the default implementation
> - ChRootedFileSystem for proper resolution of viewfs mount table
> - FilterFileSystem to avoid calling the default implementation
> Only then the same method in the target ({{LocalFileSystem}} in this case)
> will be called. Hdfs does not suffer from the same flaw since it applies
> umask in all cases, regardless of what version of {{mkdirs()}} was called.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16582) LocalFileSystem's mkdirs() does not work as expected under viewfs.
[
https://issues.apache.org/jira/browse/HADOOP-16582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kihwal Lee updated HADOOP-16582:
Attachment: HADOOP-16582.1.patch
> LocalFileSystem's mkdirs() does not work as expected under viewfs.
> --
>
> Key: HADOOP-16582
> URL: https://issues.apache.org/jira/browse/HADOOP-16582
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Major
> Attachments: HADOOP-16582.1.patch, HADOOP-16582.patch
>
>
> When {{mkdirs(Path)}} is called against {{LocalFileSystem}}, the
> implementation in {{RawLocalFileSystem}} is called and the directory
> permission is determined by the umask. However, if it is under
> {{ViewFileSystem}}, the default implementation in {{FileSystem}} is called
> and this causes explicit {{chmod()}} to 0777.
> The {{mkdirs(Path)}} method needs to be overriden in
> - ViewFileSystem to avoid calling the default implementation
> - ChRootedFileSystem for proper resolution of viewfs mount table
> - FilterFileSystem to avoid calling the default implementation
> Only then the same method in the target ({{LocalFileSystem}} in this case)
> will be called. Hdfs does not suffer from the same flaw since it applies
> umask in all cases, regardless of what version of {{mkdirs()}} was called.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1468: HDDS-2138. OM bucket operations do not add up
elek closed pull request #1468: HDDS-2138. OM bucket operations do not add up URL: https://github.com/apache/hadoop/pull/1468 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
[
https://issues.apache.org/jira/browse/HADOOP-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ranith Sardar updated HADOOP-16585:
---
Description:
{code:java}
// id would be same for multiple file, so it may occur file not found exception
private void write() throws IOException {
String dirName = dirs.get(r.nextInt(dirs.size()));
Path file = new Path(dirName, hostname+id);
fc.delete(file, true);
..
}
{code}
> [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating
> file LoadGenerator#write
> ---
>
> Key: HADOOP-16585
> URL: https://issues.apache.org/jira/browse/HADOOP-16585
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ranith Sardar
>Assignee: Ranith Sardar
>Priority: Major
>
> {code:java}
> // id would be same for multiple file, so it may occur file not found
> exception
> private void write() throws IOException {
> String dirName = dirs.get(r.nextInt(dirs.size()));
> Path file = new Path(dirName, hostname+id);
>
> fc.delete(file, true);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16585) [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write
Ranith Sardar created HADOOP-16585: -- Summary: [Tool:NNloadGeneratorMR] Multiple threads are using same id for creating file LoadGenerator#write Key: HADOOP-16585 URL: https://issues.apache.org/jira/browse/HADOOP-16585 Project: Hadoop Common Issue Type: Bug Reporter: Ranith Sardar Assignee: Ranith Sardar -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS
hadoop-yetus removed a comment on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS URL: https://github.com/apache/hadoop/pull/1402#issuecomment-532391837 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 50 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 1 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1344 | trunk passed | | +1 | compile | 36 | trunk passed | | +1 | checkstyle | 27 | trunk passed | | +1 | mvnsite | 40 | trunk passed | | +1 | shadedclient | 851 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 28 | trunk passed | | 0 | spotbugs | 68 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 66 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 37 | the patch passed | | +1 | compile | 31 | the patch passed | | +1 | javac | 31 | the patch passed | | +1 | checkstyle | 23 | the patch passed | | +1 | mvnsite | 38 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 935 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 26 | the patch passed | | +1 | findbugs | 72 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 84 | hadoop-aws in the patch passed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 3836 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1402 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux faf14cf15b01 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / eefe9bc | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/5/testReport/ | | Max. process+thread count | 440 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/5/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS
hadoop-yetus removed a comment on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS URL: https://github.com/apache/hadoop/pull/1402#issuecomment-531284055 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1142 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1157 | trunk passed | | +1 | compile | 31 | trunk passed | | +1 | checkstyle | 24 | trunk passed | | +1 | mvnsite | 36 | trunk passed | | +1 | shadedclient | 784 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 25 | trunk passed | | 0 | spotbugs | 59 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 57 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 33 | the patch passed | | +1 | compile | 27 | the patch passed | | +1 | javac | 27 | the patch passed | | +1 | checkstyle | 18 | the patch passed | | +1 | mvnsite | 31 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 830 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 22 | the patch passed | | +1 | findbugs | 60 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 72 | hadoop-aws in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 4473 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1402 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 16b5370caf95 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 95010a4 | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/4/testReport/ | | Max. process+thread count | 350 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/4/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on issue #568: HADOOP-15691 Add PathCapabilities to FS
and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#issuecomment-532696623
all checksums are legit, will fix
```
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java:30:import
java.util.Locale;:8: Unused import - java.util.Locale. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java:50:import
org.apache.hadoop.fs.impl.PathCapabilitiesSupport;:8: Unused import -
org.apache.hadoop.fs.impl.PathCapabilitiesSupport. [UnusedImports]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FsLinkResolution.java:76:
T apply(final AbstractFileSystem fs, final Path path):13: Redundant 'final'
modifier. [RedundantModifier]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/FsLinkResolution.java:76:
T apply(final AbstractFileSystem fs, final Path path):42: Redundant 'final'
modifier. [RedundantModifier]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/PathCapabilitiesSupport.java:30:@InterfaceAudience.Private:
Missing a Javadoc comment. [JavadocType]
./hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/PathCapabilitiesSupport.java:30:@InterfaceAudience.Private:1:
Utility classes should not have a public or default constructor.
[HideUtilityClassConstructor]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:34:import
org.apache.hadoop.fs.CommonPathCapabilities;:8: Unused import -
org.apache.hadoop.fs.CommonPathCapabilities. [UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:125:import
static
org.apache.hadoop.fs.impl.PathCapabilitiesSupport.validatePathCapabilityArgs;:15:
Unused import -
org.apache.hadoop.fs.impl.PathCapabilitiesSupport.validatePathCapabilityArgs.
[UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/DfsPathCapabilities.java:29:public
final class DfsPathCapabilities {: Missing a Javadoc comment. [JavadocType]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:49:import
java.util.Locale;:8: Unused import - java.util.Locale. [UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:66:import
org.apache.hadoop.fs.CommonPathCapabilities;:8: Unused import -
org.apache.hadoop.fs.CommonPathCapabilities. [UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:80:import
org.apache.hadoop.fs.PathCapabilities;:8: Unused import -
org.apache.hadoop.fs.PathCapabilities. [UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:140:import
static
org.apache.hadoop.fs.impl.PathCapabilitiesSupport.validatePathCapabilityArgs;:15:
Unused import -
org.apache.hadoop.fs.impl.PathCapabilitiesSupport.validatePathCapabilityArgs.
[UnusedImports]
./hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java:89:import
java.util.Locale;:8: Unused import - java.util.Locale. [UnusedImports]
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/select/SelectConstants.java:53:
public static final String S3_SELECT_CAPABILITY =
"fs.s3a.capability.select.sql";: Line is longer than 80 characters (found 83).
[LineLength]
./hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/ITestCommitOperations.java:29:import
org.junit.Assume;:8: Unused import - org.junit.Assume. [UnusedImports]
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
hadoop-yetus removed a comment on issue #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities URL: https://github.com/apache/hadoop/pull/568#issuecomment-486043659 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 74 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 8 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 329 | Maven dependency ordering for branch | | +1 | mvninstall | 1442 | trunk passed | | +1 | compile | 1490 | trunk passed | | +1 | checkstyle | 222 | trunk passed | | +1 | mvnsite | 391 | trunk passed | | +1 | shadedclient | 1484 | branch has no errors when building and testing our client artifacts. | | +1 | findbugs | 500 | trunk passed | | +1 | javadoc | 252 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | +1 | mvninstall | 195 | the patch passed | | +1 | compile | 1052 | the patch passed | | -1 | javac | 1052 | root generated 2 new + 1496 unchanged - 0 fixed = 1498 total (was 1496) | | -0 | checkstyle | 148 | root: The patch generated 11 new + 598 unchanged - 0 fixed = 609 total (was 598) | | +1 | mvnsite | 290 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 746 | patch has no errors when building and testing our client artifacts. | | +1 | findbugs | 468 | the patch passed | | +1 | javadoc | 211 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 560 | hadoop-common in the patch passed. | | +1 | unit | 119 | hadoop-hdfs-client in the patch passed. | | +1 | unit | 271 | hadoop-hdfs-httpfs in the patch passed. | | +1 | unit | 299 | hadoop-aws in the patch passed. | | +1 | unit | 84 | hadoop-azure in the patch passed. | | +1 | unit | 60 | hadoop-azure-datalake in the patch passed. | | +1 | asflicense | 44 | The patch does not generate ASF License warnings. | | | | 10465 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-568/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/568 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 37f7dbdddc93 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar 18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / fec9bf4 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/4/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/4/artifact/out/diff-checkstyle-root.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/4/testReport/ | | Max. process+thread count | 1446 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-azure-datalake U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/4/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
hadoop-yetus removed a comment on issue #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities URL: https://github.com/apache/hadoop/pull/568#issuecomment-485056931 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 23 | Docker mode activated. | ||| _ Prechecks _ | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 8 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 111 | Maven dependency ordering for branch | | +1 | mvninstall | 1198 | trunk passed | | +1 | compile | 1117 | trunk passed | | +1 | checkstyle | 173 | trunk passed | | +1 | mvnsite | 324 | trunk passed | | +1 | shadedclient | 1268 | branch has no errors when building and testing our client artifacts. | | +1 | findbugs | 452 | trunk passed | | +1 | javadoc | 217 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 24 | Maven dependency ordering for patch | | +1 | mvninstall | 215 | the patch passed | | +1 | compile | 1008 | the patch passed | | -1 | javac | 1008 | root generated 2 new + 1496 unchanged - 0 fixed = 1498 total (was 1496) | | -0 | checkstyle | 148 | root: The patch generated 9 new + 583 unchanged - 0 fixed = 592 total (was 583) | | +1 | mvnsite | 282 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 741 | patch has no errors when building and testing our client artifacts. | | +1 | findbugs | 439 | the patch passed | | +1 | javadoc | 214 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 511 | hadoop-common in the patch passed. | | +1 | unit | 117 | hadoop-hdfs-client in the patch passed. | | +1 | unit | 271 | hadoop-hdfs-httpfs in the patch passed. | | +1 | unit | 289 | hadoop-aws in the patch passed. | | +1 | unit | 93 | hadoop-azure in the patch passed. | | +1 | unit | 59 | hadoop-azure-datalake in the patch passed. | | +1 | asflicense | 43 | The patch does not generate ASF License warnings. | | | | 9135 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-568/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/568 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux a1ccd482e36f 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 317fcba | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_191 | | findbugs | v3.1.0-RC1 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/3/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/3/artifact/out/diff-checkstyle-root.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/3/testReport/ | | Max. process+thread count | 1580 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-azure-datalake U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-568/3/console | | Powered by | Apache Yetus 0.9.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-16484) S3A to warn or fail if S3Guard is disabled
[ https://issues.apache.org/jira/browse/HADOOP-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota reassigned HADOOP-16484: --- Assignee: Gabor Bota > S3A to warn or fail if S3Guard is disabled > -- > > Key: HADOOP-16484 > URL: https://issues.apache.org/jira/browse/HADOOP-16484 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Minor > > A seemingly recurrent problem with s3guard is "people who think S3Guard is > turned on but really it isn't" > It's not immediately obvious this is the case, and the fact S3Guard is off > tends to surface after some intermittent failure has actually been detected. > Propose: add a configuration parameter which chooses what to do when an S3A > FS is instantiated without S3Guard > * silent : today; do nothing. > * inform: log FS is instantiated without s3guard > * warn: Warn that data may be at risk in workflows > * fail > deployments could then choose which level of reaction they want. I'd make the > default "inform" for now; any on-prem object store deployment should switch > to silent, and if you really want strictness, fail is the ultimate option -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work stopped] (HADOOP-16520) race condition in DDB table init and waiting threads
[ https://issues.apache.org/jira/browse/HADOOP-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-16520 stopped by Gabor Bota. --- > race condition in DDB table init and waiting threads > > > Key: HADOOP-16520 > URL: https://issues.apache.org/jira/browse/HADOOP-16520 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.0 >Reporter: Steve Loughran >Assignee: Gabor Bota >Priority: Major > > s3guard threads waiting for table creation completion can be scheduled before > the creating thread, look for the version marker and then fail. > window will be sleep times in AWS SDK Table.waitForActive(); -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS
hadoop-yetus commented on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS URL: https://github.com/apache/hadoop/pull/1402#issuecomment-532681274 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1200 | trunk passed | | +1 | compile | 32 | trunk passed | | +1 | checkstyle | 24 | trunk passed | | +1 | mvnsite | 36 | trunk passed | | +1 | shadedclient | 784 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 27 | trunk passed | | 0 | spotbugs | 59 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 56 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 32 | the patch passed | | +1 | compile | 26 | the patch passed | | +1 | javac | 26 | the patch passed | | +1 | checkstyle | 18 | the patch passed | | +1 | mvnsite | 31 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 844 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 23 | the patch passed | | +1 | findbugs | 61 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 70 | hadoop-aws in the patch passed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 3465 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1402 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux f402829dea9e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 419dd0f | | Default Java | 1.8.0_222 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/testReport/ | | Max. process+thread count | 320 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1462: HDDS-2141. Missing total number of operations
adoroszlai commented on issue #1462: HDDS-2141. Missing total number of operations URL: https://github.com/apache/hadoop/pull/1462#issuecomment-532666465 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS
steveloughran commented on issue #1402: HADOOP-16547. make sure that s3guard prune sets up the FS URL: https://github.com/apache/hadoop/pull/1402#issuecomment-532659681 added tests; tested s3 ireland This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1451: HDDS-2134. OM Metrics graphs include empty request type
adoroszlai commented on issue #1451: HDDS-2134. OM Metrics graphs include empty request type URL: https://github.com/apache/hadoop/pull/1451#issuecomment-532657058 Thanks @elek for reviewing and committing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1451: HDDS-2134. OM Metrics graphs include empty request type
elek closed pull request #1451: HDDS-2134. OM Metrics graphs include empty request type URL: https://github.com/apache/hadoop/pull/1451 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on issue #1341: HDDS-2022. Add additional freon tests
elek commented on issue #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341#issuecomment-532649343 Merged to the trunk. Thanks @adoroszlai and @arp7 the reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1341: HDDS-2022. Add additional freon tests
elek closed pull request #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB
hadoop-yetus commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB URL: https://github.com/apache/hadoop/pull/1332#issuecomment-532628681 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 162 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1293 | trunk passed | | +1 | compile | 34 | trunk passed | | +1 | checkstyle | 29 | trunk passed | | +1 | mvnsite | 40 | trunk passed | | +1 | shadedclient | 809 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 31 | trunk passed | | 0 | spotbugs | 73 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 69 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 38 | the patch passed | | +1 | compile | 32 | the patch passed | | -1 | javac | 32 | hadoop-tools_hadoop-aws generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15) | | +1 | checkstyle | 21 | hadoop-tools/hadoop-aws: The patch generated 0 new + 34 unchanged - 1 fixed = 34 total (was 35) | | +1 | mvnsite | 38 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 869 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 27 | the patch passed | | +1 | findbugs | 68 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 86 | hadoop-aws in the patch passed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 3791 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1332 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b6fdcb367432 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e97f0f1 | | Default Java | 1.8.0_222 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/8/artifact/out/diff-compile-javac-hadoop-tools_hadoop-aws.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/8/testReport/ | | Max. process+thread count | 411 (vs. ulimit of 5500) | | modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1332/8/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1365: HDDS-1949. Missing or error-prone test cleanup
hadoop-yetus commented on issue #1365: HDDS-1949. Missing or error-prone test cleanup URL: https://github.com/apache/hadoop/pull/1365#issuecomment-532627112 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 96 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 9 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 34 | hadoop-ozone in trunk failed. | | -1 | compile | 20 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 70 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 942 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 155 | trunk passed | | 0 | spotbugs | 181 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 25 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 32 | hadoop-ozone in the patch failed. | | -1 | compile | 23 | hadoop-ozone in the patch failed. | | -1 | javac | 23 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 50 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 740 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 152 | the patch passed | | -1 | findbugs | 24 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 289 | hadoop-hdds in the patch passed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 31 | The patch does not generate ASF License warnings. | | | | 3479 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1365 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 19d9100fb3c5 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e97f0f1 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/branch-compile-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/patch-compile-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/testReport/ | | Max. process+thread count | 398 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/integration-test U: hadoop-ozone/integration-test | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1365/6/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on issue #1465: HDDS-2143. Rename classes under package org.apache.hadoop.utils.
elek commented on issue #1465: HDDS-2143. Rename classes under package org.apache.hadoop.utils. URL: https://github.com/apache/hadoop/pull/1465#issuecomment-532626462 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1462: HDDS-2141. Missing total number of operations
hadoop-yetus commented on issue #1462: HDDS-2141. Missing total number of operations URL: https://github.com/apache/hadoop/pull/1462#issuecomment-532625264 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 31 | hadoop-ozone in trunk failed. | | -1 | compile | 20 | hadoop-ozone in trunk failed. | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1004 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 187 | trunk passed | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 34 | hadoop-ozone in the patch failed. | | -1 | jshint | 93 | The patch generated 1392 new + 2737 unchanged - 0 fixed = 4129 total (was 2737) | | -1 | compile | 26 | hadoop-ozone in the patch failed. | | -1 | javac | 27 | hadoop-ozone in the patch failed. | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 802 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 176 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 262 | hadoop-hdds in the patch passed. | | -1 | unit | 31 | hadoop-ozone in the patch failed. | | +1 | asflicense | 36 | The patch does not generate ASF License warnings. | | | | 3123 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1462 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient jshint | | uname | Linux c8863d1e4b09 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e97f0f1 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/branch-compile-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | jshint | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/diff-patch-jshint.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/patch-compile-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/testReport/ | | Max. process+thread count | 440 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1462/2/console | | versions | git=2.7.4 maven=3.3.9 jshint=2.10.2 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1468: HDDS-2138. OM bucket operations do not add up
hadoop-yetus commented on issue #1468: HDDS-2138. OM bucket operations do not add up URL: https://github.com/apache/hadoop/pull/1468#issuecomment-532624988 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 36 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 29 | hadoop-ozone in trunk failed. | | -1 | compile | 21 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 66 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 831 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 151 | trunk passed | | 0 | spotbugs | 166 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 26 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 32 | hadoop-ozone in the patch failed. | | -1 | compile | 23 | hadoop-ozone in the patch failed. | | -1 | javac | 23 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 34 | hadoop-ozone: The patch generated 6 new + 348 unchanged - 6 fixed = 354 total (was 354) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 628 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 151 | the patch passed | | -1 | findbugs | 26 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 245 | hadoop-hdds in the patch passed. | | -1 | unit | 27 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 3107 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1468 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0d962ce20f02 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e97f0f1 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/branch-compile-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/testReport/ | | Max. process+thread count | 629 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1468/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional
[jira] [Commented] (HADOOP-16580) Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-16580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932284#comment-16932284
]
Hadoop QA commented on HADOOP-16580:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 37m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 51s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 43s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 116 unchanged - 0 fixed = 117 total (was 116) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 21s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
50s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}134m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:39e82acc485 |
| JIRA Issue | HADOOP-16580 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12980586/HADOOP-16580.002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 9625b145ffcc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 01d7924 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16532/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16532/testReport/ |
| Max. process+thread count | 1494 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16532/console |
| Powered by | Apache
[jira] [Commented] (HADOOP-16580) Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-16580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932280#comment-16932280
]
Gergely Pollak commented on HADOOP-16580:
-
Hi [~adam.antal] thank you for the patch!
The method on line 70 of UnreliableInterface.java is annotated as idempotent
{code:java}
@Idempotent
void failsWithAccessControlExceptionEightTimes()
{code}
But it's implementation is a bit less idempotent than it should be:
{code:java}
if (failsWithAccessControlExceptionInvocationCount++ < 8) {
throw new AccessControlException();
}
{code}
I'd suggest to remove the annotation. Also the method name is a bit misleading,
because one would think it's called when it failed 8 times, but actually it
checks for the failures, and throws exception when it's the 8th failure.
> Disable retry of FailoverOnNetworkExceptionRetry in case of
> AccessControlException
> --
>
> Key: HADOOP-16580
> URL: https://issues.apache.org/jira/browse/HADOOP-16580
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Major
> Attachments: HADOOP-16580.001.patch, HADOOP-16580.002.patch
>
>
> HADOOP-14982 handled the case where a SaslException is thrown. The issue
> still persists, since the exception that is thrown is an
> *AccessControlException* because user has no kerberos credentials.
> My suggestion is that we should add this case as well to
> {{FailoverOnNetworkExceptionRetry}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16582) LocalFileSystem's mkdirs() does not work as expected under viewfs.
[
https://issues.apache.org/jira/browse/HADOOP-16582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932278#comment-16932278
]
Steve Loughran commented on HADOOP-16582:
-
LGTM
+1 pending fixing the checkstyle warnings.
> LocalFileSystem's mkdirs() does not work as expected under viewfs.
> --
>
> Key: HADOOP-16582
> URL: https://issues.apache.org/jira/browse/HADOOP-16582
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Kihwal Lee
>Assignee: Kihwal Lee
>Priority: Major
> Attachments: HADOOP-16582.patch
>
>
> When {{mkdirs(Path)}} is called against {{LocalFileSystem}}, the
> implementation in {{RawLocalFileSystem}} is called and the directory
> permission is determined by the umask. However, if it is under
> {{ViewFileSystem}}, the default implementation in {{FileSystem}} is called
> and this causes explicit {{chmod()}} to 0777.
> The {{mkdirs(Path)}} method needs to be overriden in
> - ViewFileSystem to avoid calling the default implementation
> - ChRootedFileSystem for proper resolution of viewfs mount table
> - FilterFileSystem to avoid calling the default implementation
> Only then the same method in the target ({{LocalFileSystem}} in this case)
> will be called. Hdfs does not suffer from the same flaw since it applies
> umask in all cases, regardless of what version of {{mkdirs()}} was called.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB
steveloughran commented on issue #1332: HADOOP-16445. Allow separate custom signing algorithms for S3 and DDB URL: https://github.com/apache/hadoop/pull/1332#issuecomment-532618818 > Had forgotten how difficult it is to get a patch into Hadoop. Are there any published coding standards or templates for IntelliJ / Eclipse that you're aware of? not AFAIK. Spark has some automated checks for import ordering but as there's no IDE config which works 100% of the time there either, also a PITA. But as it is 100% consistent, it is probably better at avoiding a lot of backport-merge-hell, which imports always seem to generate This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
