[GitHub] [hadoop] bshashikant closed pull request #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis.
bshashikant closed pull request #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis. URL: https://github.com/apache/hadoop/pull/1318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant opened a new pull request #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis.
bshashikant opened a new pull request #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis. URL: https://github.com/apache/hadoop/pull/1318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vivekratnavel commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun…
vivekratnavel commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun… URL: https://github.com/apache/hadoop/pull/1339#issuecomment-524208593 The integration test failure is not related to this patch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek opened a new pull request #1341: HDDS-2022. Add additional freon tests
elek opened a new pull request #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341 Freon is a generic load generator tool for ozone (ozone freon) which supports multiple generation pattern. As of now only the random-key-generator is implemented which uses ozone rpc client. It would be great to add additional tests: * Test key generation via s3 interface * Test key generation via the hadoop fs interface * Test key reads (validation) * Test OM with direct RPC calls See: https://issues.apache.org/jira/browse/HDDS-2022 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis.
hadoop-yetus commented on issue #1318: HDDS-1753. Datanode unable to find chunk while replication data using ratis. URL: https://github.com/apache/hadoop/pull/1318#issuecomment-524234061 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 45 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 5 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for branch | | +1 | mvninstall | 588 | trunk passed | | +1 | compile | 359 | trunk passed | | +1 | checkstyle | 67 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 850 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 150 | trunk passed | | 0 | spotbugs | 419 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 606 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for patch | | -1 | mvninstall | 292 | hadoop-ozone in the patch failed. | | -1 | compile | 231 | hadoop-ozone in the patch failed. | | -1 | javac | 231 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 33 | hadoop-hdds: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) | | -0 | checkstyle | 35 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 652 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 152 | the patch passed | | -1 | findbugs | 356 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 290 | hadoop-hdds in the patch passed. | | -1 | unit | 324 | hadoop-ozone in the patch failed. | | +1 | asflicense | 38 | The patch does not generate ASF License warnings. | | | | 5676 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1318 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0afc6ecbfc32 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / bd7baea | | Default Java | 1.8.0_212 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/diff-checkstyle-hadoop-hdds.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/testReport/ | | Max. process+thread count | 1345 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-hdds/container-service hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1318/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1338: HDDS-1808. TestRatisPipelineCreateAndDestory times out
adoroszlai commented on issue #1338: HDDS-1808. TestRatisPipelineCreateAndDestory times out URL: https://github.com/apache/hadoop/pull/1338#issuecomment-524234992 Thanks @bharatviswa504 for reviewing and committing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vivekratnavel opened a new pull request #1342: HDDS-2023. Fix rat check failures in trunk
vivekratnavel opened a new pull request #1342: HDDS-2023. Fix rat check failures in trunk URL: https://github.com/apache/hadoop/pull/1342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vivekratnavel commented on issue #1342: HDDS-2023. Fix rat check failures in trunk
vivekratnavel commented on issue #1342: HDDS-2023. Fix rat check failures in trunk URL: https://github.com/apache/hadoop/pull/1342#issuecomment-524239400 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vivekratnavel commented on issue #1342: HDDS-2023. Fix rat check failures in trunk
vivekratnavel commented on issue #1342: HDDS-2023. Fix rat check failures in trunk URL: https://github.com/apache/hadoop/pull/1342#issuecomment-524240693 @elek @anuengineer Please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun…
elek closed pull request #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun… URL: https://github.com/apache/hadoop/pull/1339 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun…
elek commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun… URL: https://github.com/apache/hadoop/pull/1339#issuecomment-524253368 (FTR: I fixed the whitespace issues of hadoop.css durging the commit) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai opened a new pull request #1343: HDDS-2024. Fix rat.sh problems
adoroszlai opened a new pull request #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343 ## What changes were proposed in this pull request? Fix problems in `rat.sh`: 1. `grep: warning: recursive search of stdin` due to missing search target (which works with GNU grep, but not with some others) 2. `cat: target/rat-aggregated.txt: No such file or directory` due mismatch of `REPORT_FILE` value vs. literal 3. typo in `REPORT_FILE` value Also: use `-s` test instead of `cat` (probably cheaper and safer). https://issues.apache.org/jira/browse/HDDS-2024 ## How was this patch tested? ``` $ ./hadoop-ozone/dev-support/checks/rat.sh ... [INFO] Build failures were ignored. hadoop-hdds/common/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-hdds/common/src/main/java/org/apache/hadoop/utils/db/cache/CacheResult.java hadoop-ozone/dist/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/dist/src/main/compose/ozones3-haproxy/haproxy-conf/haproxy.cfg hadoop-ozone/integration-test/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientForAclAuditLog.java hadoop-ozone/common/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/BooleanBiFunction.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/bucket/TestS3BucketDeleteResponse.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/response/s3/multipart/TestS3MultipartUploadAbortResponse.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadCompleteRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/TestS3MultipartUploadAbortRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/multipart/S3MultipartUploadCompleteResponse.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMKeyPurgeResponse.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/ratis/utils/OzoneManagerDoubleBufferHelper.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadCompleteRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyPurgeRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/acl/OMBucketSetAclRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/acl/OMBucketRemoveAclRequest.java hadoop-ozone/ozone-manager/target/rat.txt: !? /Users/adoroszlai/src/hadoop/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/volume/acl/OMVolumeAclRequest.java $ echo $? 1 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems
adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343#issuecomment-524258110 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems
adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343#issuecomment-524258894 @elek please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1342: HDDS-2023. Fix rat check failures in trunk
elek closed pull request #1342: HDDS-2023. Fix rat check failures in trunk URL: https://github.com/apache/hadoop/pull/1342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek closed pull request #1343: HDDS-2024. Fix rat.sh problems
elek closed pull request #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project
elek commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project URL: https://github.com/apache/hadoop/pull/1340#issuecomment-524265095 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems
adoroszlai commented on issue #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343#issuecomment-524265113 Thanks @elek for the review and commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] elek commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project
elek commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project URL: https://github.com/apache/hadoop/pull/1340#issuecomment-524265117 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1341: HDDS-2022. Add additional freon tests
hadoop-yetus commented on issue #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341#issuecomment-524270997 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 628 | trunk passed | | +1 | compile | 368 | trunk passed | | +1 | checkstyle | 71 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 960 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 179 | trunk passed | | 0 | spotbugs | 518 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 766 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 584 | the patch passed | | +1 | compile | 398 | the patch passed | | +1 | javac | 398 | the patch passed | | +1 | checkstyle | 78 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 762 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 175 | the patch passed | | -1 | findbugs | 537 | hadoop-ozone generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 383 | hadoop-hdds in the patch passed. | | -1 | unit | 3403 | hadoop-ozone in the patch failed. | | +1 | asflicense | 55 | The patch does not generate ASF License warnings. | | | | 9835 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Possible null pointer dereference of volume in org.apache.hadoop.ozone.freon.BaseFreonGenerator.ensureVolumeAndBucketExist(OzoneConfiguration, String, String) on exception path Dereferenced at BaseFreonGenerator.java:volume in org.apache.hadoop.ozone.freon.BaseFreonGenerator.ensureVolumeAndBucketExist(OzoneConfiguration, String, String) on exception path Dereferenced at BaseFreonGenerator.java:[line 259] | | | Unused field:OzoneClientKeyValidator.java | | | Unused field:OzoneClientKeyValidator.java | | | Dead store to configuration in org.apache.hadoop.ozone.freon.S3KeyGenerator.call() At S3KeyGenerator.java:org.apache.hadoop.ozone.freon.S3KeyGenerator.call() At S3KeyGenerator.java:[line 78] | | Failed junit tests | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.hdds.scm.pipeline.TestNode2PipelineMap | | | hadoop.hdds.scm.pipeline.TestNodeFailure | | | hadoop.ozone.client.rpc.Test2WayCommitInRatis | | | hadoop.ozone.om.TestOzoneManagerHA | | | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.ozone.TestMiniOzoneCluster | | | hadoop.hdds.scm.pipeline.TestSCMRestart | | | hadoop.ozone.TestMiniChaosOzoneCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1341 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux a7e864d92109 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / bd7baea | | Default Java | 1.8.0_222 | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/1/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/1/testReport/ | | Max. process+thread count | 3993 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/tools U: hadoop-ozone/tools | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git S
[GitHub] [hadoop] hadoop-yetus commented on issue #1343: HDDS-2024. Fix rat.sh problems
hadoop-yetus commented on issue #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343#issuecomment-524274637 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 44 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 754 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1023 | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 617 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 783 | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 | unit | 103 | hadoop-hdds in the patch passed. | | +1 | unit | 297 | hadoop-ozone in the patch passed. | | +1 | asflicense | 42 | The patch does not generate ASF License warnings. | | | | 3890 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1343 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs | | uname | Linux d8d2be37227c 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / b4a95a2 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/1/testReport/ | | Max. process+thread count | 334 (vs. ulimit of 5500) | | modules | C: hadoop-ozone U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/1/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16430) S3AFilesystem.delete to incrementally update s3guard with deletions
[ https://issues.apache.org/jira/browse/HADOOP-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914182#comment-16914182 ] Steve Loughran commented on HADOOP-16430: - Highlight: coding this has thrown up an issue with the current implementation. the recursive delete only asks S3 for files, not DDB. This ensures that incomplete DDB stores are not a problem, so that deletion does mostly recover from any problems. However, it relies on LIST being complete, which we know is untrue as it is eventually consistent. Proposed: after deleting all files, we do a recursive list of what is left in S3Guard and delete those too. That way files we know about but which the list missed will still get cleaned up. > S3AFilesystem.delete to incrementally update s3guard with deletions > --- > > Key: HADOOP-16430 > URL: https://issues.apache.org/jira/browse/HADOOP-16430 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.2.0, 3.3.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Attachments: Screenshot 2019-07-16 at 22.08.31.png > > > Currently S3AFilesystem.delete() only updates the delete at the end of a > paged delete operation. This makes it slow when there are many thousands of > files to delete ,and increases the window of vulnerability to failures > Preferred > * after every bulk DELETE call is issued to S3, queue the (async) delete of > all entries in that post. > * at the end of the delete, await the completion of these operations. > * inside S3AFS, also do the delete across threads, so that different HTTPS > connections can be used. > This should maximise DDB throughput against tables which aren't IO limited. > When executed against small IOP limited tables, the parallel DDB DELETE > batches will trigger a lot of throttling events; we should make sure these > aren't going to trigger failures -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1342: HDDS-2023. Fix rat check failures in trunk
hadoop-yetus commented on issue #1342: HDDS-2023. Fix rat check failures in trunk URL: https://github.com/apache/hadoop/pull/1342#issuecomment-524278564 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 5 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 73 | Maven dependency ordering for branch | | +1 | mvninstall | 610 | trunk passed | | +1 | compile | 344 | trunk passed | | +1 | checkstyle | 64 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 790 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 156 | trunk passed | | 0 | spotbugs | 408 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 602 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for patch | | +1 | mvninstall | 546 | the patch passed | | +1 | compile | 364 | the patch passed | | +1 | javac | 364 | the patch passed | | +1 | checkstyle | 70 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 667 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 152 | the patch passed | | +1 | findbugs | 616 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 282 | hadoop-hdds in the patch passed. | | -1 | unit | 1729 | hadoop-ozone in the patch failed. | | +1 | asflicense | 45 | The patch does not generate ASF License warnings. | | | | 7328 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.om.TestOzoneManagerHA | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1342/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1342 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux fbadb4fcee90 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / bd7baea | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1342/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1342/1/testReport/ | | Max. process+thread count | 5213 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/common hadoop-ozone/dist hadoop-ozone/integration-test hadoop-ozone/ozone-manager U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1342/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1343: HDDS-2024. Fix rat.sh problems
hadoop-yetus commented on issue #1343: HDDS-2024. Fix rat.sh problems URL: https://github.com/apache/hadoop/pull/1343#issuecomment-524279594 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 45 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 1 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 594 | trunk passed | | +1 | mvnsite | 1 | trunk passed | | +1 | shadedclient | 764 | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 551 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 727 | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 | unit | 97 | hadoop-hdds in the patch passed. | | +1 | unit | 278 | hadoop-ozone in the patch passed. | | +1 | asflicense | 36 | The patch does not generate ASF License warnings. | | | | 3281 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1343 | | Optional Tests | dupname asflicense mvnsite unit shellcheck shelldocs | | uname | Linux 96a2de459f81 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d3fe993 | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/2/testReport/ | | Max. process+thread count | 446 (vs. ulimit of 5500) | | modules | C: hadoop-ozone U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1343/2/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-16469) Typo in s3a committers.md doc
[ https://issues.apache.org/jira/browse/HADOOP-16469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16469. - Fix Version/s: 3.3.0 Resolution: Fixed > Typo in s3a committers.md doc > - > > Key: HADOOP-16469 > URL: https://issues.apache.org/jira/browse/HADOOP-16469 > Project: Hadoop Common > Issue Type: Sub-task > Components: documentation, fs/s3 >Affects Versions: 3.1.0 >Reporter: Steve Loughran >Priority: Minor > Fix For: 3.3.0 > > > There's a typo in the s3a committers doc'; PR to fix it filed -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on a change in pull request #1341: HDDS-2022. Add additional freon tests
adoroszlai commented on a change in pull request #1341: HDDS-2022. Add
additional freon tests
URL: https://github.com/apache/hadoop/pull/1341#discussion_r317094661
##
File path:
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/BaseFreonGenerator.java
##
@@ -0,0 +1,325 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.hadoop.ozone.freon;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ipc.Client;
+import org.apache.hadoop.ipc.ProtobufRpcEngine;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.client.OzoneClient;
+import org.apache.hadoop.ozone.client.OzoneClientFactory;
+import org.apache.hadoop.ozone.client.OzoneVolume;
+import org.apache.hadoop.ozone.om.exceptions.OMException;
+import org.apache.hadoop.ozone.om.exceptions.OMException.ResultCodes;
+import
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB;
+import org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolPB;
+import org.apache.hadoop.security.UserGroupInformation;
+
+import com.codahale.metrics.ConsoleReporter;
+import com.codahale.metrics.MetricRegistry;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.commons.lang3.RandomStringUtils;
+import org.apache.ratis.protocol.ClientId;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import picocli.CommandLine.Option;
+import picocli.CommandLine.ParentCommand;
+
+/**
+ * Base class for simplified performance tests.
+ */
+public class BaseFreonGenerator {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(BaseFreonGenerator.class);
+
+ private static final int CHECK_INTERVAL_MILLIS = 1000;
+
+ private static final String DIGEST_ALGORITHM = "MD5";
+
+ private static final Pattern ENV_VARIABLE_IN_PATTERN =
+ Pattern.compile("__(.+?)__");
+
+ @ParentCommand
+ private Freon freonCommand;
+
+ @Option(names = {"-n", "--number-of-tests"},
+ description = "Number of the generated objects.",
+ defaultValue = "1000")
+ private long testNo = 1000;
+
+ @Option(names = {"-t", "--threads", "--thread"},
+ description = "Number of threads used to execute",
+ defaultValue = "10")
+ private int threadNo;
+
+ @Option(names = {"-f", "--fail-at-end"},
+ description = "If turned on, all the tasks will be executed even if "
+ + "there are failures.")
+ private boolean failAtEnd;
+
+ @Option(names = {"-p", "--prefix"},
+ description = "Unique identifier of the test execution. Usually used as"
+ + " a prefix of the generated object names. If empty, a random name"
+ + " will be generated",
+ defaultValue = "")
+ private String prefix = "";
+
+ private MetricRegistry metrics = new MetricRegistry();
+
+ private ExecutorService executor;
+
+ private AtomicLong successCounter;
+
+ private AtomicLong failureCounter;
+
+ private long startTime;
+
+ private PathSchema pathSchema;
+
+ /**
+ * The main logic to execute a test generator.
+ *
+ * @param provider creates the new steps to execute.
+ */
+ public void runTests(TaskProvider provider) {
+
+executor = Executors.newFixedThreadPool(threadNo);
+
+ProgressBar progressBar =
+new ProgressBar(System.out, testNo, successCounter::get);
+progressBar.start();
+
+startTime = System.currentTimeMillis();
+//schedule the execution of all the tasks.
+for (int i = 0; i < testNo; i++) {
+ executor.execute(() -> {
+try {
+
+ //in case of an other failed test, we shouldn't execute more tasks.
+ if (!failAtEnd && failureCounter.get() > 0) {
+return;
+ }
+
+ provider.executeNextTask(successCounter.get());
Review comment:
```suggestion
[GitHub] [hadoop] adoroszlai commented on a change in pull request #1341: HDDS-2022. Add additional freon tests
adoroszlai commented on a change in pull request #1341: HDDS-2022. Add
additional freon tests
URL: https://github.com/apache/hadoop/pull/1341#discussion_r317094938
##
File path:
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/BaseFreonGenerator.java
##
@@ -0,0 +1,325 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.hadoop.ozone.freon;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ipc.Client;
+import org.apache.hadoop.ipc.ProtobufRpcEngine;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.client.OzoneClient;
+import org.apache.hadoop.ozone.client.OzoneClientFactory;
+import org.apache.hadoop.ozone.client.OzoneVolume;
+import org.apache.hadoop.ozone.om.exceptions.OMException;
+import org.apache.hadoop.ozone.om.exceptions.OMException.ResultCodes;
+import
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB;
+import org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolPB;
+import org.apache.hadoop.security.UserGroupInformation;
+
+import com.codahale.metrics.ConsoleReporter;
+import com.codahale.metrics.MetricRegistry;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.commons.lang3.RandomStringUtils;
+import org.apache.ratis.protocol.ClientId;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import picocli.CommandLine.Option;
+import picocli.CommandLine.ParentCommand;
+
+/**
+ * Base class for simplified performance tests.
+ */
+public class BaseFreonGenerator {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(BaseFreonGenerator.class);
+
+ private static final int CHECK_INTERVAL_MILLIS = 1000;
+
+ private static final String DIGEST_ALGORITHM = "MD5";
+
+ private static final Pattern ENV_VARIABLE_IN_PATTERN =
+ Pattern.compile("__(.+?)__");
+
+ @ParentCommand
+ private Freon freonCommand;
+
+ @Option(names = {"-n", "--number-of-tests"},
+ description = "Number of the generated objects.",
+ defaultValue = "1000")
+ private long testNo = 1000;
+
+ @Option(names = {"-t", "--threads", "--thread"},
+ description = "Number of threads used to execute",
+ defaultValue = "10")
+ private int threadNo;
+
+ @Option(names = {"-f", "--fail-at-end"},
+ description = "If turned on, all the tasks will be executed even if "
+ + "there are failures.")
+ private boolean failAtEnd;
+
+ @Option(names = {"-p", "--prefix"},
+ description = "Unique identifier of the test execution. Usually used as"
+ + " a prefix of the generated object names. If empty, a random name"
+ + " will be generated",
+ defaultValue = "")
+ private String prefix = "";
+
+ private MetricRegistry metrics = new MetricRegistry();
+
+ private ExecutorService executor;
+
+ private AtomicLong successCounter;
Review comment:
I think we need a separate `attemptCounter` to uniquely identify tasks and
avoid duplicate keys by multiple threads.
```suggestion
private AtomicLong successCounter;
private AtomicLong attemptCounter;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on a change in pull request #1341: HDDS-2022. Add additional freon tests
adoroszlai commented on a change in pull request #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341#discussion_r317092258 ## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OzoneClientKeyValidator.java ## @@ -0,0 +1,105 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with this + * work for additional information regarding copyright ownership. The ASF + * licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.hadoop.ozone.freon; + +import java.io.InputStream; +import java.security.MessageDigest; +import java.util.concurrent.Callable; + +import org.apache.hadoop.hdds.cli.HddsVersionProvider; +import org.apache.hadoop.hdds.conf.OzoneConfiguration; +import org.apache.hadoop.ozone.client.OzoneBucket; +import org.apache.hadoop.ozone.client.OzoneClient; +import org.apache.hadoop.ozone.client.OzoneClientFactory; + +import com.codahale.metrics.Timer; +import org.apache.commons.io.IOUtils; +import picocli.CommandLine.Command; +import picocli.CommandLine.Option; + +/** + * Data generator tool test om performance. + */ +@Command(name = "ockv", +aliases = "ozone-client-key-validator", +description = "Generate keys with the help of the ozone clients.", Review comment: ```suggestion description = "Validate keys with the help of the ozone clients.", ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] adoroszlai commented on a change in pull request #1341: HDDS-2022. Add additional freon tests
adoroszlai commented on a change in pull request #1341: HDDS-2022. Add
additional freon tests
URL: https://github.com/apache/hadoop/pull/1341#discussion_r317095088
##
File path:
hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/BaseFreonGenerator.java
##
@@ -0,0 +1,325 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.hadoop.ozone.freon;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.net.InetSocketAddress;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import org.apache.hadoop.hdds.conf.OzoneConfiguration;
+import org.apache.hadoop.ipc.Client;
+import org.apache.hadoop.ipc.ProtobufRpcEngine;
+import org.apache.hadoop.ipc.RPC;
+import org.apache.hadoop.net.NetUtils;
+import org.apache.hadoop.ozone.OmUtils;
+import org.apache.hadoop.ozone.client.OzoneClient;
+import org.apache.hadoop.ozone.client.OzoneClientFactory;
+import org.apache.hadoop.ozone.client.OzoneVolume;
+import org.apache.hadoop.ozone.om.exceptions.OMException;
+import org.apache.hadoop.ozone.om.exceptions.OMException.ResultCodes;
+import
org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB;
+import org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolPB;
+import org.apache.hadoop.security.UserGroupInformation;
+
+import com.codahale.metrics.ConsoleReporter;
+import com.codahale.metrics.MetricRegistry;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.commons.lang3.RandomStringUtils;
+import org.apache.ratis.protocol.ClientId;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import picocli.CommandLine.Option;
+import picocli.CommandLine.ParentCommand;
+
+/**
+ * Base class for simplified performance tests.
+ */
+public class BaseFreonGenerator {
+
+ private static final Logger LOG =
+ LoggerFactory.getLogger(BaseFreonGenerator.class);
+
+ private static final int CHECK_INTERVAL_MILLIS = 1000;
+
+ private static final String DIGEST_ALGORITHM = "MD5";
+
+ private static final Pattern ENV_VARIABLE_IN_PATTERN =
+ Pattern.compile("__(.+?)__");
+
+ @ParentCommand
+ private Freon freonCommand;
+
+ @Option(names = {"-n", "--number-of-tests"},
+ description = "Number of the generated objects.",
+ defaultValue = "1000")
+ private long testNo = 1000;
+
+ @Option(names = {"-t", "--threads", "--thread"},
+ description = "Number of threads used to execute",
+ defaultValue = "10")
+ private int threadNo;
+
+ @Option(names = {"-f", "--fail-at-end"},
+ description = "If turned on, all the tasks will be executed even if "
+ + "there are failures.")
+ private boolean failAtEnd;
+
+ @Option(names = {"-p", "--prefix"},
+ description = "Unique identifier of the test execution. Usually used as"
+ + " a prefix of the generated object names. If empty, a random name"
+ + " will be generated",
+ defaultValue = "")
+ private String prefix = "";
+
+ private MetricRegistry metrics = new MetricRegistry();
+
+ private ExecutorService executor;
+
+ private AtomicLong successCounter;
+
+ private AtomicLong failureCounter;
+
+ private long startTime;
+
+ private PathSchema pathSchema;
+
+ /**
+ * The main logic to execute a test generator.
+ *
+ * @param provider creates the new steps to execute.
+ */
+ public void runTests(TaskProvider provider) {
+
+executor = Executors.newFixedThreadPool(threadNo);
+
+ProgressBar progressBar =
+new ProgressBar(System.out, testNo, successCounter::get);
+progressBar.start();
+
+startTime = System.currentTimeMillis();
+//schedule the execution of all the tasks.
+for (int i = 0; i < testNo; i++) {
+ executor.execute(() -> {
+try {
+
+ //in case of an other failed test, we shouldn't execute more tasks.
+ if (!failAtEnd && failureCounter.get() > 0) {
+return;
+ }
+
+ provider.executeNextTask(successCounter.get());
+ successCounter.incrementAndGet();
+
[jira] [Commented] (HADOOP-16268) Allow custom wrapped exception to be thrown by server if RPC call queue is filled up
[
https://issues.apache.org/jira/browse/HADOOP-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914353#comment-16914353
]
Erik Krogen commented on HADOOP-16268:
--
+1 on the latest patch
> Allow custom wrapped exception to be thrown by server if RPC call queue is
> filled up
>
>
> Key: HADOOP-16268
> URL: https://issues.apache.org/jira/browse/HADOOP-16268
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Attachments: HADOOP-16268.001.patch, HADOOP-16268.002.patch,
> HADOOP-16268.003.patch, HADOOP-16268.004.patch
>
>
> In the current implementation of callqueue manager,
> "CallQueueOverflowException" exceptions are always wrapping
> "RetriableException". Through configs servers should be allowed to throw
> custom exceptions based on new use cases.
> In CallQueueManager.java for backoff the below is done
> {code:java}
> // ideally this behavior should be controllable too.
> private void throwBackoff() throws IllegalStateException {
> throw CallQueueOverflowException.DISCONNECT;
> }
> {code}
> Since CallQueueOverflowException only wraps RetriableException clients would
> end up hitting the same server for retries. In use cases that router supports
> these overflowed requests could be handled by another router that shares the
> same state thus distributing load across a cluster of routers better. In the
> absence of any custom exception, current behavior should be supported.
> In CallQueueOverflowException class a new Standby exception wrap should be
> created. Something like the below
> {code:java}
>static final CallQueueOverflowException KEEPALIVE =
> new CallQueueOverflowException(
> new RetriableException(TOO_BUSY),
> RpcStatusProto.ERROR);
> static final CallQueueOverflowException DISCONNECT =
> new CallQueueOverflowException(
> new RetriableException(TOO_BUSY + " - disconnecting"),
> RpcStatusProto.FATAL);
> static final CallQueueOverflowException DISCONNECT2 =
> new CallQueueOverflowException(
> new StandbyException(TOO_BUSY + " - disconnecting"),
> RpcStatusProto.FATAL);
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sodonnel opened a new pull request #1344: HDDS-1982 Extend SCMNodeManager to support decommission and maintenance states
sodonnel opened a new pull request #1344: HDDS-1982 Extend SCMNodeManager to support decommission and maintenance states URL: https://github.com/apache/hadoop/pull/1344 Remove the existing decommission states from the protobuf definition. At this stage, this PR is really a test to see if the build passes with these states removed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sodonnel commented on issue #1344: HDDS-1982 Extend SCMNodeManager to support decommission and maintenance states
sodonnel commented on issue #1344: HDDS-1982 Extend SCMNodeManager to support decommission and maintenance states URL: https://github.com/apache/hadoop/pull/1344#issuecomment-524376220 /label ozone This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] nandakumar131 commented on a change in pull request #1331: HDDS-2002. Update documentation for 0.4.1 release.
nandakumar131 commented on a change in pull request #1331: HDDS-2002. Update
documentation for 0.4.1 release.
URL: https://github.com/apache/hadoop/pull/1331#discussion_r317218266
##
File path: hadoop-hdds/docs/content/interface/OzoneFS.md
##
@@ -36,7 +36,7 @@ ozone sh volume create /volume
ozone sh bucket create /volume/bucket
{{< /highlight >}}
-Once this is created, please make sure that bucket exists via the listVolume
or listBucket commands.
+Once this is created, please make sure that bucket exists via the _list
volume_ or _list bucket_ commands.
Review comment:
Thanks, will fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on a change in pull request #1329: HDDS-738. Removing REST protocol support from OzoneClient
hadoop-yetus commented on a change in pull request #1329: HDDS-738. Removing REST protocol support from OzoneClient URL: https://github.com/apache/hadoop/pull/1329#discussion_r317219351 ## File path: hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/TestOzoneClientProducer.java ## @@ -17,31 +17,30 @@ */ package org.apache.hadoop.ozone.s3; -import org.apache.hadoop.hdds.conf.OzoneConfiguration; -import org.apache.hadoop.ozone.OzoneConfigKeys; -import org.apache.hadoop.ozone.om.OMConfigKeys; -import org.apache.hadoop.test.LambdaTestUtils; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; -import org.mockito.Mockito; - import javax.ws.rs.container.ContainerRequestContext; import javax.ws.rs.core.MultivaluedHashMap; import javax.ws.rs.core.MultivaluedMap; import javax.ws.rs.core.UriInfo; - import java.io.IOException; import java.net.URI; import java.util.Arrays; import java.util.Collection; +import org.apache.hadoop.hdds.conf.OzoneConfiguration; +import org.apache.hadoop.ozone.OzoneConfigKeys; +import org.apache.hadoop.ozone.om.OMConfigKeys; +import org.apache.hadoop.test.LambdaTestUtils; + import static org.apache.hadoop.ozone.s3.AWSAuthParser.AUTHORIZATION_HEADER; import static org.apache.hadoop.ozone.s3.AWSAuthParser.CONTENT_MD5; import static org.apache.hadoop.ozone.s3.AWSAuthParser.CONTENT_TYPE; import static org.apache.hadoop.ozone.s3.AWSAuthParser.HOST_HEADER; import static org.apache.hadoop.ozone.s3.AWSAuthParser.X_AMAZ_DATE; import static org.apache.hadoop.ozone.s3.AWSAuthParser.X_AMZ_CONTENT_SHA256; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; Review comment: whitespace:end of line This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1329: HDDS-738. Removing REST protocol support from OzoneClient
hadoop-yetus commented on issue #1329: HDDS-738. Removing REST protocol support from OzoneClient URL: https://github.com/apache/hadoop/pull/1329#issuecomment-524387964 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 4 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 45 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 71 | Maven dependency ordering for branch | | +1 | mvninstall | 593 | trunk passed | | +1 | compile | 382 | trunk passed | | +1 | checkstyle | 81 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 851 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 177 | trunk passed | | 0 | spotbugs | 441 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 645 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 40 | Maven dependency ordering for patch | | +1 | mvninstall | 539 | the patch passed | | +1 | compile | 380 | the patch passed | | +1 | javac | 101 | hadoop-hdds in the patch passed. | | +1 | javac | 279 | hadoop-ozone generated 0 new + 5 unchanged - 3 fixed = 5 total (was 8) | | -0 | checkstyle | 43 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 1 | There were no new shellcheck issues. | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 6 | The patch has no ill-formed XML file. | | +1 | shadedclient | 759 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 99 | hadoop-ozone generated 1 new + 27 unchanged - 1 fixed = 28 total (was 28) | | +1 | findbugs | 768 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 347 | hadoop-hdds in the patch passed. | | -1 | unit | 1465 | hadoop-ozone in the patch failed. | | +1 | asflicense | 48 | The patch does not generate ASF License warnings. | | | | 7743 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestScmSafeMode | | | hadoop.ozone.TestOzoneConfigurationFields | | | hadoop.ozone.om.TestOmAcls | | | hadoop.ozone.TestStorageContainerManager | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.om.TestOmBlockVersioning | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1329 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml shellcheck shelldocs | | uname | Linux 8ee0952ffb28 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / c92de82 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/artifact/out/diff-checkstyle-hadoop-ozone.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/artifact/out/whitespace-eol.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/artifact/out/diff-javadoc-javadoc-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/testReport/ | | Max. process+thread count | 4504 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone hadoop-ozone/client hadoop-ozone/common hadoop-ozone/datanode hadoop-ozone/dist hadoop-ozone/integration-test hadoop-ozone/ozone-manager hadoop-ozone/ozonefs hadoop-ozone/s3gateway hadoop-ozone/tools U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/3/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For quer
[GitHub] [hadoop] nandakumar131 merged pull request #1310: HDDS-1978. Create helper script to run blockade tests.
nandakumar131 merged pull request #1310: HDDS-1978. Create helper script to run blockade tests. URL: https://github.com/apache/hadoop/pull/1310 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] nandakumar131 commented on issue #1326: HDDS-1898. GrpcReplicationService#download cannot replicate the container.
nandakumar131 commented on issue #1326: HDDS-1898. GrpcReplicationService#download cannot replicate the container. URL: https://github.com/apache/hadoop/pull/1326#issuecomment-524388620 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1341: HDDS-2022. Add additional freon tests
hadoop-yetus commented on issue #1341: HDDS-2022. Add additional freon tests URL: https://github.com/apache/hadoop/pull/1341#issuecomment-524399971 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 98 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -1 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 650 | trunk passed | | +1 | compile | 362 | trunk passed | | +1 | checkstyle | 65 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 823 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 152 | trunk passed | | 0 | spotbugs | 420 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 609 | trunk passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 549 | the patch passed | | +1 | compile | 362 | the patch passed | | +1 | javac | 362 | the patch passed | | +1 | checkstyle | 69 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 671 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 152 | the patch passed | | -1 | findbugs | 426 | hadoop-ozone generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 219 | hadoop-hdds in the patch failed. | | -1 | unit | 2613 | hadoop-ozone in the patch failed. | | +1 | asflicense | 43 | The patch does not generate ASF License warnings. | | | | 8234 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Possible null pointer dereference of volume in org.apache.hadoop.ozone.freon.BaseFreonGenerator.ensureVolumeAndBucketExist(OzoneConfiguration, String, String) on exception path Dereferenced at BaseFreonGenerator.java:volume in org.apache.hadoop.ozone.freon.BaseFreonGenerator.ensureVolumeAndBucketExist(OzoneConfiguration, String, String) on exception path Dereferenced at BaseFreonGenerator.java:[line 263] | | | Unused field:OzoneClientKeyValidator.java | | | Unused field:OzoneClientKeyValidator.java | | | Dead store to configuration in org.apache.hadoop.ozone.freon.S3KeyGenerator.call() At S3KeyGenerator.java:org.apache.hadoop.ozone.freon.S3KeyGenerator.call() At S3KeyGenerator.java:[line 78] | | | Unused field:SameKeyReader.java | | | Unused field:SameKeyReader.java | | Failed junit tests | hadoop.ozone.container.ozoneimpl.TestOzoneContainer | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.TestStorageContainerManager | | | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.ozone.client.rpc.Test2WayCommitInRatis | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1341 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 03d0f0374a50 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / c92de82 | | Default Java | 1.8.0_222 | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/testReport/ | | Max. process+thread count | 4876 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/tools U: hadoop-ozone/tools | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1341/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated me
[GitHub] [hadoop] hadoop-yetus commented on issue #1331: HDDS-2002. Update documentation for 0.4.1 release.
hadoop-yetus commented on issue #1331: HDDS-2002. Update documentation for 0.4.1 release. URL: https://github.com/apache/hadoop/pull/1331#issuecomment-524402581 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 51 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 597 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1393 | branch has no errors when building and testing our client artifacts. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 553 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 675 | patch has no errors when building and testing our client artifacts. | ||| _ Other Tests _ | | +1 | asflicense | 42 | The patch does not generate ASF License warnings. | | | | 2883 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1331/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1331 | | Optional Tests | dupname asflicense mvnsite | | uname | Linux 2debd257db3c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 20064b6 | | Max. process+thread count | 411 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/docs U: hadoop-hdds/docs | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1331/4/console | | versions | git=2.7.4 maven=3.3.9 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #1334: HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly
goiri merged pull request #1334: HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly URL: https://github.com/apache/hadoop/pull/1334 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16438) Introduce a config to control SSL Channel mode in Azure DataLake Store Gen1
[ https://issues.apache.org/jira/browse/HADOOP-16438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914518#comment-16914518 ] Steve Loughran commented on HADOOP-16438: - yetus is being a bit restrictive, to the extent of not testing patches from people it doesn't trust -sorry. If you reinstate the PR we can review it there and then i can kick yetus into action can you submit the move to a newer SDK as a separate patch? > Introduce a config to control SSL Channel mode in Azure DataLake Store Gen1 > --- > > Key: HADOOP-16438 > URL: https://issues.apache.org/jira/browse/HADOOP-16438 > Project: Hadoop Common > Issue Type: Task > Components: fs/adl >Affects Versions: 2.9.0 >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Attachments: HADOOP-16438.001.patch > > > Currently there is no user control possible on the SSL channel mode used for > server connections. It will try to connect using SSLChannelMode.OpenSSL and > default to SSLChannelMode.Default_JSE when there is any issue. > A new config is needed to toggle the choice if any issues are observed with > OpenSSL. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16255) ChecksumFS.Make FileSystem.rename(path, path, options) doesn't rename checksum
[ https://issues.apache.org/jira/browse/HADOOP-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16255: Affects Version/s: 2.8.5 > ChecksumFS.Make FileSystem.rename(path, path, options) doesn't rename checksum > -- > > Key: HADOOP-16255 > URL: https://issues.apache.org/jira/browse/HADOOP-16255 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.8.5, 3.1.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > ChecksumFS doesn't override FilterFS rename/3, so doesn't rename the checksum > with the file. > As a result, if a file is renamed over an existing file using rename(src, > dest, OVERWRITE) the renamed file will be considered to have an invalid > checksum -the old one is picked up instead. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16255) ChecksumFS.Make FileSystem.rename(path, path, options) doesn't rename checksum
[ https://issues.apache.org/jira/browse/HADOOP-16255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914524#comment-16914524 ] Steve Loughran commented on HADOOP-16255: - I see your issue. # It should just be matter of overriding the rename/3 call and having it move the CRC tool; this is good to preserve the checksum too. # patches with tests welcome... # we're never going to do another 2.7.x release so saying it is affected isn't going to make anyone care. I've tagged 2.8.5 as affected too # now. can you upgrade spark's dependencies? please? > ChecksumFS.Make FileSystem.rename(path, path, options) doesn't rename checksum > -- > > Key: HADOOP-16255 > URL: https://issues.apache.org/jira/browse/HADOOP-16255 > Project: Hadoop Common > Issue Type: Bug > Components: fs >Affects Versions: 2.8.5, 3.1.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > > ChecksumFS doesn't override FilterFS rename/3, so doesn't rename the checksum > with the file. > As a result, if a file is renamed over an existing file using rename(src, > dest, OVERWRITE) the renamed file will be considered to have an invalid > checksum -the old one is picked up instead. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pingsutw commented on issue #1328: HDDS-1998. TestSecureContainerServer#testClientServerRatisGrpc is fai…
pingsutw commented on issue #1328: HDDS-1998. TestSecureContainerServer#testClientServerRatisGrpc is fai… URL: https://github.com/apache/hadoop/pull/1328#issuecomment-524410689 @bshashikant thanks for the reply, we can't get **_ContainerCommandResponseProto response_** from _**finalClient.sendCommand(request)**_ due to IOException. but we can know StorageContainerException from ex.getMessage() how could I get StorageContainerException instance from IOException This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #955: HDFS-14478: Add libhdfs APIs for openFile
steveloughran commented on issue #955: HDFS-14478: Add libhdfs APIs for openFile URL: https://github.com/apache/hadoop/pull/955#issuecomment-524411216 I dont see any tests for trying to open a file which doesnt exist. When you do that. the failure is postponed until the completion is evaluated, so you need to add a test there to make sure the failure is escalated properly This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16193) add extra S3A MPU test to see what happens if a file is created during the MPU
[ https://issues.apache.org/jira/browse/HADOOP-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914530#comment-16914530 ] Steve Loughran commented on HADOOP-16193: - thx > add extra S3A MPU test to see what happens if a file is created during the MPU > -- > > Key: HADOOP-16193 > URL: https://issues.apache.org/jira/browse/HADOOP-16193 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Minor > Fix For: 3.1.3 > > > Proposed extra test for the S3A MPU: if you create and then delete a file > while an MPU is in progress, when you finally complete the MPU the new data > is present. > This verifies that the other FS operations don't somehow cancel the > in-progress upload, and that eventual consistency brings the latest value out. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path
hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path URL: https://github.com/apache/hadoop/pull/269#issuecomment-524042022 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 35 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 23 | Maven dependency ordering for branch | | +1 | mvninstall | 1184 | trunk passed | | +1 | compile | 1082 | trunk passed | | +1 | checkstyle | 163 | trunk passed | | +1 | mvnsite | 160 | trunk passed | | +1 | shadedclient | 1131 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 141 | trunk passed | | 0 | spotbugs | 199 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 340 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 24 | Maven dependency ordering for patch | | +1 | mvninstall | 113 | the patch passed | | +1 | compile | 1096 | the patch passed | | +1 | javac | 1096 | the patch passed | | -0 | checkstyle | 151 | root: The patch generated 7 new + 126 unchanged - 0 fixed = 133 total (was 126) | | +1 | mvnsite | 173 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 738 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 123 | the patch passed | | +1 | findbugs | 341 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 565 | hadoop-common in the patch passed. | | -1 | unit | 75 | hadoop-hdfs in the patch failed. | | +1 | asflicense | 49 | The patch does not generate ASF License warnings. | | | | 7636 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-269/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/269 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 39113425507a 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 63c295e | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/7/artifact/out/diff-checkstyle-root.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/7/testReport/ | | Max. process+thread count | 1492 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/7/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path
hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path URL: https://github.com/apache/hadoop/pull/269#issuecomment-522181715 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 46 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 24 | Maven dependency ordering for branch | | +1 | mvninstall | 1080 | trunk passed | | +1 | compile | 1047 | trunk passed | | +1 | checkstyle | 130 | trunk passed | | +1 | mvnsite | 132 | trunk passed | | +1 | shadedclient | 945 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 116 | trunk passed | | 0 | spotbugs | 159 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 268 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 104 | the patch passed | | +1 | compile | 1007 | the patch passed | | +1 | javac | 1007 | the patch passed | | -0 | checkstyle | 141 | root: The patch generated 7 new + 126 unchanged - 0 fixed = 133 total (was 126) | | +1 | mvnsite | 135 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 658 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 114 | the patch passed | | +1 | findbugs | 301 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 503 | hadoop-common in the patch failed. | | -1 | unit | 4885 | hadoop-hdfs in the patch failed. | | +1 | asflicense | 48 | The patch does not generate ASF License warnings. | | | | 11716 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ha.TestZKFailoverController | | | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.hdfs.qjournal.server.TestJournalNodeSync | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/269 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b1d0d8ce5a92 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 8943e13 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/artifact/out/diff-checkstyle-root.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/testReport/ | | Max. process+thread count | 4274 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/5/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path
hadoop-yetus removed a comment on issue #269: HADOOP-14217. Support colon in Hadoop Path URL: https://github.com/apache/hadoop/pull/269#issuecomment-523196673 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for branch | | +1 | mvninstall | 1072 | trunk passed | | +1 | compile | 1015 | trunk passed | | +1 | checkstyle | 129 | trunk passed | | +1 | mvnsite | 131 | trunk passed | | +1 | shadedclient | 973 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 113 | trunk passed | | 0 | spotbugs | 170 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 281 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 109 | the patch passed | | +1 | compile | 1042 | the patch passed | | +1 | javac | 1042 | the patch passed | | -0 | checkstyle | 136 | root: The patch generated 7 new + 126 unchanged - 0 fixed = 133 total (was 126) | | +1 | mvnsite | 141 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 649 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 121 | the patch passed | | +1 | findbugs | 296 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 545 | hadoop-common in the patch passed. | | -1 | unit | 4813 | hadoop-hdfs in the patch failed. | | +1 | asflicense | 48 | The patch does not generate ASF License warnings. | | | | 11689 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency | | | hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer | | | hadoop.hdfs.tools.TestDFSZKFailoverController | | | hadoop.hdfs.server.balancer.TestBalancer | | | hadoop.hdfs.server.namenode.TestFsck | | | hadoop.hdfs.server.balancer.TestBalancerRPCDelay | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-269/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/269 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0eb8da722e92 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 4cb22cd | | Default Java | 1.8.0_212 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/6/artifact/out/diff-checkstyle-root.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/6/testReport/ | | Max. process+thread count | 4472 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-269/6/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1302: HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE
steveloughran commented on issue #1302: HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE URL: https://github.com/apache/hadoop/pull/1302#issuecomment-524412807 :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
hadoop-yetus removed a comment on issue #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden URL: https://github.com/apache/hadoop/pull/533#issuecomment-519354077 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 12 | https://github.com/apache/hadoop/pull/533 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/533 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/7/console | | versions | git=2.7.4 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317264692
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
##
@@ -1362,4 +1364,24 @@ public boolean equals(Object other) {
new CompletableFuture<>(), () -> open(path, bufferSize));
}
+ /**
+ * Return the base capabilities of the filesystems
+ * may override to declare different behavior.
Review comment:
deleted second line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317265105
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
##
@@ -1362,4 +1364,24 @@ public boolean equals(Object other) {
new CompletableFuture<>(), () -> open(path, bufferSize));
}
+ /**
+ * Return the base capabilities of the filesystems
+ * may override to declare different behavior.
+ * @param path path to query the capability of.
+ * @param capability string to query the stream support for.
+ * @return true if the capability is supported under that part of the FS.
+ * @throws IOException on failure
Review comment:
I've just deleted the whole javadoc; the interface says what it does.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317265320
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonPathCapabilities.java
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs;
+
+/**
+ * Common path capabilities.
+ */
+public final class CommonPathCapabilities {
+
+ public CommonPathCapabilities() {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities URL: https://github.com/apache/hadoop/pull/568#discussion_r317265700 ## File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFs.java ## @@ -446,4 +446,9 @@ public BlockStoragePolicySpi getStoragePolicy(final Path src) return myFs.openFileWithOptions(path, mandatoryKeys, options, bufferSize); } + public boolean hasPathCapability(final Path path, Review comment: well caught. fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317266100
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathCapabilities.java
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs;
+
+import java.io.IOException;
+
+/**
+ * The Path counterpoint to {@link StreamCapabilities}; a query to see if,
+ * a FileSystem/FileContext instance has a specific capability under the given
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317266970
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathCapabilities.java
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs;
+
+import java.io.IOException;
+
+/**
+ * The Path counterpoint to {@link StreamCapabilities}; a query to see if,
+ * a FileSystem/FileContext instance has a specific capability under the given
+ * path.
+ */
+public interface PathCapabilities {
+
+ /**
+ * Probe for a filesystem instance offering a specific capability under the
Review comment:
done;
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317268134
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathCapabilities.java
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs;
+
+import java.io.IOException;
+
+/**
+ * The Path counterpoint to {@link StreamCapabilities}; a query to see if,
+ * a FileSystem/FileContext instance has a specific capability under the given
+ * path.
+ */
+public interface PathCapabilities {
+
+ /**
+ * Probe for a filesystem instance offering a specific capability under the
+ * given path.
+ * If the function returns {@code true}, the filesystem is explicitly
+ * declaring that the capability is available.
+ * If the function returns {@code false}, it can mean one of:
+ *
+ * The capability is not known.
+ * The capability is known but it is not supported.
+ * The capability is known but the filesystem does not know if it
+ * is supported under the supplied path it.
+ *
+ * The core guarantee which a caller can rely on is: if the predicate
+ * returns true, then the specific operation/behavior can be expected to be
+ * supported.
+ * @param path path to query the capability of.
+ * @param capability non-null, non-empty string to query the path for
support.
+ * @return true if the capability is supported under that part of the FS.
+ * @throws IOException this should not be raised, except on problems
+ * resolving paths or relaying the call.
+ * @throws IllegalArgumentException invalid arguments
Review comment:
mentioned the class
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement
default acls for bucket/volume/key for OM HA code.
URL: https://github.com/apache/hadoop/pull/1315#discussion_r317268149
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java
##
@@ -205,6 +215,26 @@ public OMClientResponse
validateAndUpdateCache(OzoneManager ozoneManager,
}
+ /**
+ * Add default acls for bucket. These acls are inherited from volume
+ * default acl list.
+ * @param omBucketInfo
+ * @param omVolumeArgs
+ */
+ private void addDefaultAcls(OmBucketInfo omBucketInfo,
+ OmVolumeArgs omVolumeArgs) {
+// Add default acls from volume.
+List acls = new ArrayList<>();
+if (omBucketInfo.getAcls() != null) {
+ acls.addAll(omBucketInfo.getAcls());
+}
+omVolumeArgs.getAclMap().getDefaultAclList().forEach(
+defaultAcl -> acls.add(
Review comment:
Simple add does not merge acl properly. You can use
OzoneAclUtil.inheritDefaultAcls(acls,
omVolumeArgs.getAclMap().getDefaultAclList()) to do the merge.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317268891
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/http/AbstractHttpFileSystem.java
##
@@ -111,6 +114,21 @@ public FileStatus getFileStatus(Path path) throws
IOException {
return new FileStatus(-1, false, 1, DEFAULT_BLOCK_SIZE, 0, path);
}
+ /**
+ * Declare that this filesystem connector is always read only.
+ * {@inheritDoc}
+ */
+ @Override
+ public boolean hasPathCapability(final Path path, final String capability)
+ throws IOException {
+switch (validatePathCapabilityArgs(path, capability)) {
+case CommonPathCapabilities.FS_READ_ONLY_CONNECTOR:
+ return true;
+default:
+ return false;
Review comment:
I like the way it makes clear that this is R/O, but am happy to change it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317269946
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/impl/PathCapabilitiesSupport.java
##
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.impl;
+
+import java.util.Locale;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+import org.apache.hadoop.fs.Path;
+
+import static com.google.common.base.Preconditions.checkArgument;
+
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class PathCapabilitiesSupport {
+
+ /**
+ * Validate the arguments to {@code PathCapabilities.hadCapability()}.
+ * @param path path to query the capability of.
+ * @param capability non-null, non-empty string to query the path for
support.
+ * @return the string to use in a switch statement.
+ * @throws IllegalArgumentException if a an argument is invalid.
+ */
+ public static String validatePathCapabilityArgs(
+ final Path path, final String capability) {
+checkArgument(path != null, "null path");
+checkArgument(capability != null && !capability.isEmpty(),
+"null/empty capability");
Review comment:
ok, split the two up
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. URL: https://github.com/apache/hadoop/pull/1315#discussion_r317270565 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java ## @@ -351,28 +362,65 @@ protected OmKeyInfo createKeyInfo(@Nonnull KeyArgs keyArgs, .setDataSize(size) .setReplicationType(type) .setReplicationFactor(factor) -.setFileEncryptionInfo(encInfo); +.setFileEncryptionInfo(encInfo) +.setAcls(getAclsForKey(keyArgs, omBucketInfo, prefixManager)).build(); + } + + private List< OzoneAcl > getAclsForKey(KeyArgs keyArgs, Review comment: Can we move this to some common helper util such as OzoneAclUtil to avoid duplicate code in KeyManagerImpl#getAclsForKey? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317271234
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
##
@@ -931,6 +933,36 @@ public Path getLinkTarget(Path path) throws IOException {
return res.targetFileSystem.getLinkTarget(res.remainingPath);
}
+ /**
+ * Fail fast on known write capabilities (append, concat),
+ * forward the rest to the viewed FS.
+ * @param path path to query the capability of.
+ * @param capability string to query the stream support for.
+ * @return the capability
+ * @throws IOException if there is no resolved FS, or it raises an IOE.
+ */
+ @Override
+ public boolean hasPathCapability(Path path, String capability)
+ throws IOException {
+final Path p = makeQualified(path);
+switch (validatePathCapabilityArgs(p, capability)) {
+case CommonPathCapabilities.FS_CONCAT:
+ // concat is not supported, as it may be invoked across filesystems.
+ return false;
+default:
Review comment:
done. clarified in javadocs that this is only for concat
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. URL: https://github.com/apache/hadoop/pull/1315#discussion_r317271309 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java ## @@ -447,4 +497,31 @@ private OMClientResponse createKeyErrorResponse(@Nonnull OMMetrics omMetrics, } } + /** + * Check Acls for the ozone object. + * @param ozoneManager + * @param volume + * @param bucket + * @param key + * @param checkKeyAccess + * @throws IOException + */ + protected void checkKeyAcls(OzoneManager ozoneManager, String volume, Review comment: Suggest to refactor into two separate APIs one for checkKeyAccess and one for checkBucketAccess to avoid confusion. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dineshchitlangia opened a new pull request #1345: HDDS-2013. Add flag gdprEnabled for BucketInfo in OzoneManager proto
dineshchitlangia opened a new pull request #1345: HDDS-2013. Add flag gdprEnabled for BucketInfo in OzoneManager proto URL: https://github.com/apache/hadoop/pull/1345 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
xiaoyuyao commented on a change in pull request #1315: HDDS-1975. Implement
default acls for bucket/volume/key for OM HA code.
URL: https://github.com/apache/hadoop/pull/1315#discussion_r317273644
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/bucket/S3BucketDeleteRequest.java
##
@@ -107,12 +105,6 @@ public OMClientResponse
validateAndUpdateCache(OzoneManager ozoneManager,
OMMetadataManager omMetadataManager = ozoneManager.getMetadataManager();
OMClientResponse omClientResponse = null;
try {
- // check Acl
Review comment:
Maybe add some TODOs to support S3 ACL later?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1224: HDDS-1810. SCM command to Activate and Deactivate pipelines.
hadoop-yetus commented on issue #1224: HDDS-1810. SCM command to Activate and Deactivate pipelines. URL: https://github.com/apache/hadoop/pull/1224#issuecomment-524440397 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 152 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 54 | Maven dependency ordering for branch | | +1 | mvninstall | 853 | trunk passed | | +1 | compile | 433 | trunk passed | | +1 | checkstyle | 89 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1088 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 199 | trunk passed | | 0 | spotbugs | 519 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 754 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 37 | Maven dependency ordering for patch | | +1 | mvninstall | 688 | the patch passed | | +1 | compile | 437 | the patch passed | | +1 | cc | 437 | the patch passed | | +1 | javac | 437 | the patch passed | | +1 | checkstyle | 90 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 833 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 211 | the patch passed | | +1 | findbugs | 789 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 374 | hadoop-hdds in the patch passed. | | -1 | unit | 2346 | hadoop-ozone in the patch failed. | | +1 | asflicense | 51 | The patch does not generate ASF License warnings. | | | | 9680 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestOzoneManager | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1224/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1224 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 53efbb2436d6 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 20064b6 | | Default Java | 1.8.0_212 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1224/7/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1224/7/testReport/ | | Max. process+thread count | 5262 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/client hadoop-hdds/common hadoop-hdds/server-scm hadoop-hdds/tools hadoop-ozone/integration-test U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1224/7/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317280104
##
File path:
hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
##
@@ -1470,3 +1470,144 @@ hsync| HSYNC | Syncable | Flush
out the data in client's us
in:readahead | READAHEAD | CanSetReadahead | Set the readahead on the input
stream.
dropbehind | DROPBEHIND | CanSetDropBehind | Drop the cache.
in:unbuffer | UNBUFFER | CanUnbuffer | Reduce the buffering on the
input stream.
+
+## interface `PathCapabilities`
+
+The `PathCapabilities` provides a way to programmatically query the operations
+offered by a FileSystem or FileContext instance under a given path.
+
+```java
+public interface PathCapabilities {
+ boolean hasPathCapability(Path path, String capability)
+ throws IOException;
+}
+```
+
+There are a number of goals here:
+
+1. Allow callers to probe for optional filesystem operations without actually
+having to invoke them.
+1. Allow filesystems with their own optional per-instance features to declare
+whether or not they are active for the specific instance.
+1. Allow for fileystem connectors which work with object stores to expose the
+fundamental difference in semantics of these stores (e.g: files not visible
+until closed, file rename being `O(data)`), directory rename being non-atomic,
+etc.
+
+### Available Capabilities
+
+Capabilities are defined with a store prefix and an arbitrary (but hopefully
+meaningful) string.
+
+All custom filesystem-specific capabilities MUST be given the prefix of that
+filesystem schema. The standard schemas are:
+
+* `fs` : File system capabilities
+* `object` : Object capabilities.
+
+The exact set of operations and their names are evolving.
+
+Consult the javadocs for `org.apache.hadoop.fs.PathCapabilities` for the
+standard set of capabilities.
+
+Individual filesystems may offer their own set of capabilities which
+can be probed for. These begin with the same prefix as the filesystem schema,
+such as `hdfs:` or `s3a:`.
+
+### `boolean hasPathCapability(path, capability)`
+
+Probe for a filesystem instance offering a specific capability under the
+given path.
+
+ Postconditions
+
+```python
+if fs_supports_the_feature(path, capability):
+ return True
+else:
+ return False
+```
+
+Return: `True`, iff the specific capability is, the the best of the
Review comment:
reworked. Also moved the entire interface into its own file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hanishakoneru commented on issue #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru commented on issue #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts. URL: https://github.com/apache/hadoop/pull/1130#issuecomment-524442065 Thank you @arp7 for the review. I have opened a Jira HDDS-2027 to improve the unit test. Will merge this PR shortly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hanishakoneru merged pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts.
hanishakoneru merged pull request #1130: HDDS-1827. Load Snapshot info when OM Ratis server starts. URL: https://github.com/apache/hadoop/pull/1130 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] vivekratnavel commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun…
vivekratnavel commented on issue #1339: HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trun… URL: https://github.com/apache/hadoop/pull/1339#issuecomment-524451354 @elek Thanks for the whitespace fix and commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317292842
##
File path:
hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
##
@@ -1470,3 +1470,144 @@ hsync| HSYNC | Syncable | Flush
out the data in client's us
in:readahead | READAHEAD | CanSetReadahead | Set the readahead on the input
stream.
dropbehind | DROPBEHIND | CanSetDropBehind | Drop the cache.
in:unbuffer | UNBUFFER | CanUnbuffer | Reduce the buffering on the
input stream.
+
+## interface `PathCapabilities`
+
+The `PathCapabilities` provides a way to programmatically query the operations
+offered by a FileSystem or FileContext instance under a given path.
+
+```java
+public interface PathCapabilities {
+ boolean hasPathCapability(Path path, String capability)
+ throws IOException;
+}
+```
+
+There are a number of goals here:
+
+1. Allow callers to probe for optional filesystem operations without actually
+having to invoke them.
+1. Allow filesystems with their own optional per-instance features to declare
+whether or not they are active for the specific instance.
+1. Allow for fileystem connectors which work with object stores to expose the
+fundamental difference in semantics of these stores (e.g: files not visible
+until closed, file rename being `O(data)`), directory rename being non-atomic,
+etc.
+
+### Available Capabilities
+
+Capabilities are defined with a store prefix and an arbitrary (but hopefully
+meaningful) string.
+
+All custom filesystem-specific capabilities MUST be given the prefix of that
+filesystem schema. The standard schemas are:
+
+* `fs` : File system capabilities
+* `object` : Object capabilities.
+
+The exact set of operations and their names are evolving.
+
+Consult the javadocs for `org.apache.hadoop.fs.PathCapabilities` for the
+standard set of capabilities.
+
+Individual filesystems may offer their own set of capabilities which
+can be probed for. These begin with the same prefix as the filesystem schema,
+such as `hdfs:` or `s3a:`.
+
+### `boolean hasPathCapability(path, capability)`
+
+Probe for a filesystem instance offering a specific capability under the
+given path.
+
+ Postconditions
+
+```python
+if fs_supports_the_feature(path, capability):
+ return True
+else:
+ return False
+```
+
+Return: `True`, iff the specific capability is, the the best of the
+knowledge of the client application, available.
+
+A filesystem instance *MUST NOT* return `True` for any capability unless it is
+known to be supported by that specific instance. As a result, if a caller
+probes for a capability then it can assume that the specific feature/semantics
+are available.
+
+If the probe returns `False` then it can mean one of:
+
+1. The capability is unknown.
+1. The capability is known, but known to be unavailable on this instance.
+
+This predicate is intended to be low cost. If it requires remote calls other
+than path/link resolution, it SHOULD conclude that the availability
+of the feature is unknown and return `False`.
+
+The predicate MUST also be side-effect free.
+
+*Validity of paths*
+There is no requirement that the existence of the path must be checked;
+the parameter exists so that any filesystem which relays operations to other
+filesystems (e.g `viewfs`) can resolve and relay it to the nested filesystem.
+Consider the call to be *relatively* lightweight.
+
+Because of this, it may be that while the filesystem declares that
+it supports a capability under a path, the actual invocation of the operation
+may fail for other reasons.
+
+As an example, while a filesystem may support `append()` under a path,
+if invoked on a directory, the call may fail.
+
+That is for a path `root = new Path("/")`: the capabilities call may succeed
+
+```java
+fs.hasCapabilities(root, "fs:append") == true
+```
+
+But a subsequent call to the operation on that specific path may fail,
+because the root path is a directory
+```java
+fs.append(root)
+```
+
+The `hasCapabilities(path, capability)` probe is therefore declaring that
+the operation will not be rejected as unsupported, not that a specific
invocation
+will be considered valid.
+
+*Duration of availability*
+
+As the state of a remote store changes,so may path capabilities. This
+may be due to changes in the local state of the fileystem (e.g. symbolic links
+or mount points changing), or changes in its functionality (e.g. a feature
+becoming availaible/unavailable due to operational changes, system upgrades,
etc.)
+
+*Capabilities which must be invoked to determine availablity*
+
+Some operations may be known by the filesystem client, and believed to be
available,
+but may actually fail when invoked due to the state of the underlying
+filesystem —state which is cannot be determined except by attempting
+side-effecting operations.
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317293685
##
File path:
hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
##
@@ -1470,3 +1470,144 @@ hsync| HSYNC | Syncable | Flush
out the data in client's us
in:readahead | READAHEAD | CanSetReadahead | Set the readahead on the input
stream.
dropbehind | DROPBEHIND | CanSetDropBehind | Drop the cache.
in:unbuffer | UNBUFFER | CanUnbuffer | Reduce the buffering on the
input stream.
+
+## interface `PathCapabilities`
+
+The `PathCapabilities` provides a way to programmatically query the operations
+offered by a FileSystem or FileContext instance under a given path.
+
+```java
+public interface PathCapabilities {
+ boolean hasPathCapability(Path path, String capability)
+ throws IOException;
+}
+```
+
+There are a number of goals here:
+
+1. Allow callers to probe for optional filesystem operations without actually
+having to invoke them.
+1. Allow filesystems with their own optional per-instance features to declare
+whether or not they are active for the specific instance.
+1. Allow for fileystem connectors which work with object stores to expose the
+fundamental difference in semantics of these stores (e.g: files not visible
+until closed, file rename being `O(data)`), directory rename being non-atomic,
+etc.
+
+### Available Capabilities
+
+Capabilities are defined with a store prefix and an arbitrary (but hopefully
+meaningful) string.
+
+All custom filesystem-specific capabilities MUST be given the prefix of that
+filesystem schema. The standard schemas are:
+
+* `fs` : File system capabilities
+* `object` : Object capabilities.
+
+The exact set of operations and their names are evolving.
+
+Consult the javadocs for `org.apache.hadoop.fs.PathCapabilities` for the
+standard set of capabilities.
+
+Individual filesystems may offer their own set of capabilities which
+can be probed for. These begin with the same prefix as the filesystem schema,
+such as `hdfs:` or `s3a:`.
+
+### `boolean hasPathCapability(path, capability)`
+
+Probe for a filesystem instance offering a specific capability under the
+given path.
+
+ Postconditions
+
+```python
+if fs_supports_the_feature(path, capability):
+ return True
+else:
+ return False
+```
+
+Return: `True`, iff the specific capability is, the the best of the
+knowledge of the client application, available.
+
+A filesystem instance *MUST NOT* return `True` for any capability unless it is
+known to be supported by that specific instance. As a result, if a caller
+probes for a capability then it can assume that the specific feature/semantics
+are available.
+
+If the probe returns `False` then it can mean one of:
+
+1. The capability is unknown.
+1. The capability is known, but known to be unavailable on this instance.
+
+This predicate is intended to be low cost. If it requires remote calls other
+than path/link resolution, it SHOULD conclude that the availability
+of the feature is unknown and return `False`.
+
+The predicate MUST also be side-effect free.
+
+*Validity of paths*
+There is no requirement that the existence of the path must be checked;
+the parameter exists so that any filesystem which relays operations to other
+filesystems (e.g `viewfs`) can resolve and relay it to the nested filesystem.
+Consider the call to be *relatively* lightweight.
+
+Because of this, it may be that while the filesystem declares that
+it supports a capability under a path, the actual invocation of the operation
+may fail for other reasons.
+
+As an example, while a filesystem may support `append()` under a path,
+if invoked on a directory, the call may fail.
+
+That is for a path `root = new Path("/")`: the capabilities call may succeed
+
+```java
+fs.hasCapabilities(root, "fs:append") == true
+```
+
+But a subsequent call to the operation on that specific path may fail,
+because the root path is a directory
+```java
+fs.append(root)
+```
+
+The `hasCapabilities(path, capability)` probe is therefore declaring that
+the operation will not be rejected as unsupported, not that a specific
invocation
+will be considered valid.
+
+*Duration of availability*
+
+As the state of a remote store changes,so may path capabilities. This
+may be due to changes in the local state of the fileystem (e.g. symbolic links
+or mount points changing), or changes in its functionality (e.g. a feature
+becoming availaible/unavailable due to operational changes, system upgrades,
etc.)
+
+*Capabilities which must be invoked to determine availablity*
+
+Some operations may be known by the filesystem client, and believed to be
available,
+but may actually fail when invoked due to the state of the underlying
+filesystem —state which is cannot be determined except by attempting
+side-effecting operations.
[GitHub] [hadoop] hadoop-yetus commented on issue #1329: HDDS-738. Removing REST protocol support from OzoneClient
hadoop-yetus commented on issue #1329: HDDS-738. Removing REST protocol support from OzoneClient URL: https://github.com/apache/hadoop/pull/1329#issuecomment-524458606 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 93 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 4 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 45 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 76 | Maven dependency ordering for branch | | +1 | mvninstall | 641 | trunk passed | | +1 | compile | 397 | trunk passed | | +1 | checkstyle | 84 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 951 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 207 | trunk passed | | 0 | spotbugs | 487 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 728 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 41 | Maven dependency ordering for patch | | +1 | mvninstall | 561 | the patch passed | | +1 | compile | 392 | the patch passed | | +1 | javac | 106 | hadoop-hdds in the patch passed. | | +1 | javac | 286 | hadoop-ozone generated 0 new + 5 unchanged - 3 fixed = 5 total (was 8) | | +1 | checkstyle | 85 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | shellcheck | 1 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 7 | The patch has no ill-formed XML file. | | +1 | shadedclient | 774 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 110 | hadoop-ozone generated 1 new + 26 unchanged - 1 fixed = 27 total (was 27) | | +1 | findbugs | 733 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 421 | hadoop-hdds in the patch passed. | | -1 | unit | 2193 | hadoop-ozone in the patch failed. | | +1 | asflicense | 51 | The patch does not generate ASF License warnings. | | | | 8934 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | | hadoop.ozone.om.TestOmBlockVersioning | | | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.om.TestOmAcls | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1329 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml shellcheck shelldocs | | uname | Linux e952495ece17 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 894e230 | | Default Java | 1.8.0_222 | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/4/artifact/out/diff-javadoc-javadoc-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/4/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/4/testReport/ | | Max. process+thread count | 4777 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone hadoop-ozone/client hadoop-ozone/common hadoop-ozone/datanode hadoop-ozone/dist hadoop-ozone/integration-test hadoop-ozone/ozone-manager hadoop-ozone/ozonefs hadoop-ozone/s3gateway hadoop-ozone/tools U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1329/4/console | | versions | git=2.7.4 maven=3.3.9 shellcheck=0.4.6 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
steveloughran commented on a change in pull request #568: HADOOP-15691 Add
PathCapabilities to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317298666
##
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
##
@@ -2017,6 +2027,36 @@ public void setTestProvider(KeyProvider kp) {
testProvider = kp;
}
+ /**
+ * This filesystem's capabilities must be in sync with that of HDFS.
Review comment:
done. a bit convoluted as we need to indicate when to invoke the superclass.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)
[ https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914669#comment-16914669 ] Sean Busbey commented on HADOOP-15998: -- v4 as is doesn't work correctly. I have an amended patch about ready. > Jar validation bash scripts don't work on Windows due to platform differences > (colons in paths, \r\n) > - > > Key: HADOOP-15998 > URL: https://issues.apache.org/jira/browse/HADOOP-15998 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0, 3.3.0 > Environment: Windows 10 > Visual Studio 2017 >Reporter: Brian Grunkemeyer >Assignee: Brian Grunkemeyer >Priority: Blocker > Labels: build, windows > Attachments: HADOOP-15998.v4.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Building Hadoop fails on Windows due to a few shell scripts that make invalid > assumptions: > 1) Colon shouldn't be used to separate multiple paths in command line > parameters. Colons occur in Windows paths. > 2) Shell scripts that rely on running external tools need to deal with > carriage return - line feed differences (lines ending in \r\n, not just \n) -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)
[ https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-15998: - Attachment: HADOOP-15998.5.patch > Jar validation bash scripts don't work on Windows due to platform differences > (colons in paths, \r\n) > - > > Key: HADOOP-15998 > URL: https://issues.apache.org/jira/browse/HADOOP-15998 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.2.0, 3.3.0 > Environment: Windows 10 > Visual Studio 2017 >Reporter: Brian Grunkemeyer >Assignee: Brian Grunkemeyer >Priority: Blocker > Labels: build, windows > Attachments: HADOOP-15998.5.patch, HADOOP-15998.v4.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > Building Hadoop fails on Windows due to a few shell scripts that make invalid > assumptions: > 1) Colon shouldn't be used to separate multiple paths in command line > parameters. Colons occur in Windows paths. > 2) Shell scripts that rely on running external tools need to deal with > carriage return - line feed differences (lines ending in \r\n, not just \n) -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15998) Jar validation bash scripts don't work on Windows due to platform differences (colons in paths, \r\n)
[
https://issues.apache.org/jira/browse/HADOOP-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914675#comment-16914675
]
Sean Busbey commented on HADOOP-15998:
--
v5
- get all the artifacts passed instead of the first
- error if no artifacts
- fail if any command fails
- change reading of jar contents to avoid shell handling
- added a bash 3.1 check that we should have had before
tested locally on mac. verified that all jars are being tested, that things
fail on {{jar}} command failure, that things fail when something isn't
relocated, and that things pass correctly.
> Jar validation bash scripts don't work on Windows due to platform differences
> (colons in paths, \r\n)
> -
>
> Key: HADOOP-15998
> URL: https://issues.apache.org/jira/browse/HADOOP-15998
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.2.0, 3.3.0
> Environment: Windows 10
> Visual Studio 2017
>Reporter: Brian Grunkemeyer
>Assignee: Brian Grunkemeyer
>Priority: Blocker
> Labels: build, windows
> Attachments: HADOOP-15998.5.patch, HADOOP-15998.v4.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Building Hadoop fails on Windows due to a few shell scripts that make invalid
> assumptions:
> 1) Colon shouldn't be used to separate multiple paths in command line
> parameters. Colons occur in Windows paths.
> 2) Shell scripts that rely on running external tools need to deal with
> carriage return - line feed differences (lines ending in \r\n, not just \n)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
hadoop-yetus commented on issue #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden URL: https://github.com/apache/hadoop/pull/533#issuecomment-524483170 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 80 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 7 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 75 | Maven dependency ordering for branch | | +1 | mvninstall | 1461 | trunk passed | | +1 | compile | 1426 | trunk passed | | +1 | checkstyle | 190 | trunk passed | | +1 | mvnsite | 374 | trunk passed | | +1 | shadedclient | 1508 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 315 | trunk passed | | 0 | spotbugs | 56 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 602 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 27 | Maven dependency ordering for patch | | +1 | mvninstall | 247 | the patch passed | | +1 | compile | 1377 | the patch passed | | -1 | javac | 1377 | root generated 1 new + 1471 unchanged - 1 fixed = 1472 total (was 1472) | | -0 | checkstyle | 201 | root: The patch generated 2 new + 162 unchanged - 3 fixed = 164 total (was 165) | | +1 | mvnsite | 379 | the patch passed | | -1 | whitespace | 0 | The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | xml | 2 | The patch has no ill-formed XML file. | | +1 | shadedclient | 884 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 297 | the patch passed | | +1 | findbugs | 643 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 661 | hadoop-common in the patch passed. | | +1 | unit | 144 | hadoop-hdfs-client in the patch passed. | | +1 | unit | 41 | hadoop-openstack in the patch passed. | | +1 | unit | 105 | hadoop-aws in the patch passed. | | +1 | unit | 105 | hadoop-azure in the patch passed. | | +1 | unit | 81 | hadoop-azure-datalake in the patch passed. | | +1 | asflicense | 66 | The patch does not generate ASF License warnings. | | | | 11090 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/533 | | Optional Tests | dupname asflicense mvnsite compile javac javadoc mvninstall unit shadedclient findbugs checkstyle xml | | uname | Linux 15bf9e7d9871 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / ebef99d | | Default Java | 1.8.0_222 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/artifact/out/diff-checkstyle-root.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/testReport/ | | Max. process+thread count | 1346 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-tools/hadoop-openstack hadoop-tools/hadoop-aws hadoop-tools/hadoop-azure hadoop-tools/hadoop-azure-datalake U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-533/11/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden URL: https://github.com/apache/hadoop/pull/533#discussion_r317322338 ## File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md ## @@ -530,14 +530,21 @@ on the filesystem. ### `boolean mkdirs(Path p, FsPermission permission)` -Create a directory and all its parents +Create a directory and all its parents. Preconditions +The path must either be a directory or not exist + if exists(FS, p) and not isDir(FS, p) : raise [ParentNotDirectoryException, FileAlreadyExistsException, IOException] +No ancestor may be a file + +forall d = ancestors(FS, p) : Review comment: whitespace:end of line This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract
Tests to verify create, mkdirs and rename under a file is forbidden
URL: https://github.com/apache/hadoop/pull/533#discussion_r317322341
##
File path:
hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
##
@@ -577,14 +584,20 @@ Writing to or overwriting a directory must fail.
if isDir(FS, p) : raise {FileAlreadyExistsException,
FileNotFoundException, IOException}
+No ancestor may be a file
+
+forall d = ancestors(FS, p) :
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
hadoop-yetus commented on a change in pull request #533: HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden URL: https://github.com/apache/hadoop/pull/533#discussion_r317322335 ## File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md ## @@ -530,14 +530,21 @@ on the filesystem. ### `boolean mkdirs(Path p, FsPermission permission)` -Create a directory and all its parents +Create a directory and all its parents. Preconditions +The path must either be a directory or not exist + Review comment: whitespace:end of line This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on issue #1345: HDDS-2013. Add flag gdprEnabled for BucketInfo in OzoneManager proto
bharatviswa504 commented on issue #1345: HDDS-2013. Add flag gdprEnabled for BucketInfo in OzoneManager proto URL: https://github.com/apache/hadoop/pull/1345#issuecomment-524484667 @dineshchitlangia Just letting you know, to get CI run just post a comment /label ozone. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-519327360 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 42 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 2 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 77 | Maven dependency ordering for branch | | +1 | mvninstall | 586 | trunk passed | | +1 | compile | 351 | trunk passed | | +1 | checkstyle | 66 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 827 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 175 | trunk passed | | 0 | spotbugs | 406 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 602 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 38 | Maven dependency ordering for patch | | +1 | mvninstall | 534 | the patch passed | | +1 | compile | 365 | the patch passed | | +1 | javac | 96 | hadoop-hdds in the patch passed. | | +1 | javac | 269 | hadoop-ozone generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8) | | -0 | checkstyle | 33 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 684 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 148 | the patch passed | | +1 | findbugs | 625 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 303 | hadoop-hdds in the patch passed. | | -1 | unit | 159 | hadoop-ozone in the patch failed. | | +1 | asflicense | 41 | The patch does not generate ASF License warnings. | | | | 5823 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestS3BucketManager | | | hadoop.ozone.om.ratis.TestOzoneManagerDoubleBufferWithOMResponse | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/14/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c65235aefcce 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 3cc0ace | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/14/artifact/out/diff-checkstyle-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/14/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/14/testReport/ | | Max. process+thread count | 1387 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/ozone-recon U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/14/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-519465932 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 95 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 25 | Maven dependency ordering for branch | | +1 | mvninstall | 579 | trunk passed | | +1 | compile | 391 | trunk passed | | +1 | checkstyle | 72 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 897 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 156 | trunk passed | | 0 | spotbugs | 449 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 645 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for patch | | +1 | mvninstall | 597 | the patch passed | | +1 | compile | 349 | the patch passed | | +1 | javac | 49 | hadoop-hdds in the patch passed. | | +1 | javac | 300 | hadoop-ozone generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8) | | -0 | checkstyle | 37 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 733 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 165 | the patch passed | | +1 | findbugs | 646 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 229 | hadoop-hdds in the patch failed. | | -1 | unit | 2496 | hadoop-ozone in the patch failed. | | +1 | asflicense | 40 | The patch does not generate ASF License warnings. | | | | 8376 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.ozoneimpl.TestOzoneContainer | | | hadoop.ozone.ozShell.TestOzoneShell | | | hadoop.ozone.client.rpc.TestMultiBlockWritesWithDnFailures | | | hadoop.ozone.om.TestOzoneManagerHA | | | hadoop.ozone.TestMiniOzoneCluster | | | hadoop.ozone.client.rpc.TestOzoneRpcClient | | | hadoop.ozone.om.TestOzoneManager | | | hadoop.ozone.om.TestKeyManagerImpl | | | hadoop.ozone.client.rpc.TestSecureOzoneRpcClient | | | hadoop.ozone.om.TestScmSafeMode | | | hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures | | | hadoop.ozone.om.TestOmAcls | | | hadoop.hdds.scm.pipeline.TestRatisPipelineCreateAndDestory | | | hadoop.ozone.client.rpc.TestOzoneAtRestEncryption | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 3897c0adff94 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 00b5a27 | | Default Java | 1.8.0_212 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/artifact/out/diff-checkstyle-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/testReport/ | | Max. process+thread count | 4029 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/ozone-recon U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/15/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To u
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-522039996 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 12 | https://github.com/apache/hadoop/pull/1225 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/16/console | | versions | git=2.7.4 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-518548982 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 45 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 12 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 78 | Maven dependency ordering for branch | | +1 | mvninstall | 582 | trunk passed | | +1 | compile | 355 | trunk passed | | +1 | checkstyle | 65 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 801 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 145 | trunk passed | | 0 | spotbugs | 416 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 611 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 28 | Maven dependency ordering for patch | | +1 | mvninstall | 541 | the patch passed | | +1 | compile | 360 | the patch passed | | +1 | javac | 92 | hadoop-hdds in the patch passed. | | +1 | javac | 268 | hadoop-ozone generated 0 new + 4 unchanged - 4 fixed = 4 total (was 8) | | -0 | checkstyle | 40 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 633 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 150 | the patch passed | | +1 | findbugs | 659 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 318 | hadoop-hdds in the patch failed. | | -1 | unit | 219 | hadoop-ozone in the patch failed. | | +1 | asflicense | 38 | The patch does not generate ASF License warnings. | | | | 5879 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.common.TestDatanodeStateMachine | | | hadoop.ozone.om.TestS3BucketManager | | | hadoop.ozone.om.ratis.TestOzoneManagerDoubleBufferWithOMResponse | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 48ecf5f1386e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 1127215 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/artifact/out/diff-checkstyle-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/artifact/out/patch-unit-hadoop-hdds.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/testReport/ | | Max. process+thread count | 1407 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/common hadoop-ozone/ozone-manager hadoop-ozone/ozone-recon U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/13/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus removed a comment on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-524164842 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 16 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 23 | Maven dependency ordering for branch | | +1 | mvninstall | 568 | trunk passed | | +1 | compile | 344 | trunk passed | | +1 | checkstyle | 62 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 822 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 155 | trunk passed | | 0 | spotbugs | 443 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 640 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for patch | | +1 | mvninstall | 579 | the patch passed | | +1 | compile | 368 | the patch passed | | +1 | javac | 368 | the patch passed | | -0 | checkstyle | 35 | hadoop-ozone: The patch generated 5 new + 0 unchanged - 0 fixed = 5 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 690 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 154 | the patch passed | | +1 | findbugs | 662 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 433 | hadoop-hdds in the patch passed. | | -1 | unit | 376 | hadoop-ozone in the patch failed. | | +1 | asflicense | 35 | The patch does not generate ASF License warnings. | | | | 6245 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.request.volume.acl.TestOMVolumeAddAclRequest | | | hadoop.ozone.om.request.volume.acl.TestOMVolumeSetAclRequest | | | hadoop.ozone.om.ratis.TestOzoneManagerDoubleBufferWithOMResponse | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/18/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 2508c163bc7d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / f6af7d0 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/18/artifact/out/diff-checkstyle-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/18/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/18/testReport/ | | Max. process+thread count | 1244 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/integration-test hadoop-ozone/ozone-manager hadoop-ozone/ozone-recon U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/18/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
bharatviswa504 commented on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-524488154 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
bharatviswa504 commented on a change in pull request #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. URL: https://github.com/apache/hadoop/pull/1315#discussion_r317329818 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java ## @@ -351,28 +362,65 @@ protected OmKeyInfo createKeyInfo(@Nonnull KeyArgs keyArgs, .setDataSize(size) .setReplicationType(type) .setReplicationFactor(factor) -.setFileEncryptionInfo(encInfo); +.setFileEncryptionInfo(encInfo) +.setAcls(getAclsForKey(keyArgs, omBucketInfo, prefixManager)).build(); + } + + private List< OzoneAcl > getAclsForKey(KeyArgs keyArgs, Review comment: Discussed offline, leaving this for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bharatviswa504 commented on issue #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
bharatviswa504 commented on issue #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. URL: https://github.com/apache/hadoop/pull/1315#issuecomment-524493957 Thank You @xiaoyuyao for the review. I have addressed review comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] anuengineer closed pull request #1331: HDDS-2002. Update documentation for 0.4.1 release.
anuengineer closed pull request #1331: HDDS-2002. Update documentation for 0.4.1 release. URL: https://github.com/apache/hadoop/pull/1331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] anuengineer commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project
anuengineer commented on issue #1340: HDDS-2021. Upgrade Guava library to v26 in hdds project URL: https://github.com/apache/hadoop/pull/1340#issuecomment-524507886 @dineshchitlangia I think we have a build issue, can you please check? `[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (depcheck) @ hadoop-hdds-common --- [WARNING] Dependency convergence error for com.google.errorprone:error_prone_annotations:2.2.0 paths to dependency are: +-org.apache.hadoop:hadoop-hdds-common:0.5.0-SNAPSHOT +-com.google.errorprone:error_prone_annotations:2.2.0 and +-org.apache.hadoop:hadoop-hdds-common:0.5.0-SNAPSHOT +-com.google.guava:guava:26.0-android +-com.google.errorprone:error_prone_annotations:2.1.3` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM.
hadoop-yetus commented on issue #1225: HDDS-1909. Use new HA code for Non-HA in OM. URL: https://github.com/apache/hadoop/pull/1225#issuecomment-524522115 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 49 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 21 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 68 | Maven dependency ordering for branch | | +1 | mvninstall | 683 | trunk passed | | +1 | compile | 375 | trunk passed | | +1 | checkstyle | 72 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 833 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 157 | trunk passed | | 0 | spotbugs | 432 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 623 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 28 | Maven dependency ordering for patch | | +1 | mvninstall | 529 | the patch passed | | +1 | compile | 356 | the patch passed | | +1 | javac | 356 | the patch passed | | -0 | checkstyle | 34 | hadoop-ozone: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 630 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 155 | the patch passed | | +1 | findbugs | 632 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 275 | hadoop-hdds in the patch passed. | | -1 | unit | 333 | hadoop-ozone in the patch failed. | | +1 | asflicense | 40 | The patch does not generate ASF License warnings. | | | | 6055 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.recon.recovery.TestReconOmMetadataManagerImpl | | | hadoop.ozone.recon.spi.impl.TestOzoneManagerServiceProviderImpl | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/19/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1225 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 617da2d7d6af 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d2225c8 | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/19/artifact/out/diff-checkstyle-hadoop-ozone.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/19/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/19/testReport/ | | Max. process+thread count | 1338 (vs. ulimit of 5500) | | modules | C: hadoop-hdds/common hadoop-ozone/integration-test hadoop-ozone/ozone-manager hadoop-ozone/ozone-recon U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1225/19/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on issue #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code.
hadoop-yetus commented on issue #1315: HDDS-1975. Implement default acls for bucket/volume/key for OM HA code. URL: https://github.com/apache/hadoop/pull/1315#issuecomment-524524077 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for branch | | +1 | mvninstall | 618 | trunk passed | | +1 | compile | 376 | trunk passed | | +1 | checkstyle | 70 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 846 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 162 | trunk passed | | 0 | spotbugs | 445 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 644 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 20 | Maven dependency ordering for patch | | +1 | mvninstall | 574 | the patch passed | | +1 | compile | 368 | the patch passed | | +1 | javac | 368 | the patch passed | | +1 | checkstyle | 72 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 664 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 154 | the patch passed | | +1 | findbugs | 657 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 289 | hadoop-hdds in the patch passed. | | -1 | unit | 1984 | hadoop-ozone in the patch failed. | | +1 | asflicense | 39 | The patch does not generate ASF License warnings. | | | | 7758 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.server.TestSecureContainerServer | | | hadoop.ozone.client.rpc.TestWatchForCommit | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1315/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1315 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 9b0c75acbd59 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / d2225c8 | | Default Java | 1.8.0_212 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1315/11/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1315/11/testReport/ | | Max. process+thread count | 5101 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1315/11/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
