[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738667#comment-16738667 ] Sean Busbey commented on HDDS-965: -- I'll get a DISCUSS thread going sometime in the next week or two as I finish reflecting on things. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738637#comment-16738637 ] Anu Engineer commented on HDDS-965: --- [~busbey] Would you like to join our Ozone community meeting? We can talk about this in depth. Our next meeting is on 21st of Jan. [https://cwiki.apache.org/confluence/display/HADOOP/Ozone+Community+Calls] Thanks > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738295#comment-16738295 ] Elek, Marton commented on HDDS-965: --- Thanks the comments [~busbey]. Personally I am always +1 to a discussion. Especially as I don't think we have the final answer to the question how Ozone should be built. It was voted by the community to keep the source in the same tree but use different release lifecycle (which means slightly different build setup). But until Ozone there was no such subproject in Hadoop, we have no clear pattern which can be followed. During the previous Ozone releases we tried multiple approaches and the process was continuously improved. Some examples: Originally we used the hadoop-dist project but after a while we separated hadoop-dist and hadoop-ozone/dist (HDDS-447) Originally we used the some classpath structure as hadoop (which includes test jars + yarn jars on the classpath of the namenode), but after a while we switched to use more specific classpath definition (same HDDS-447). Originally we used the in-tree hadoop version, but now it's (almost) possible to compile ozone with multiple different hadoop version. I also agree that the build of hadoop subprojects should be as unified as possible. But Ozone is an alpha software, some good new things can be tried before moving them to the hadoop source. Personally I think that the classpath separation of HDDS-447 should be used by hdfs/yarn as well. But my feeling is that it's more risky to do such a fundamental change in hadoop side. I would prefer to try it out with ozone first and if works well we can discuss about the adoption for all the hadoop components. Similar for the testing. In Ozone we started to use docker-compose based pseudo cluster definitions and scripted smoketest/acceptance test suite based on robot framework. I think it's extremely useful not just to test Ozone as a user but to test new patches during the code review (or test the release package during the vote). I have a plan to open a jira and start similar work for hdfs/yarn as well (see ./compose and ./smoketest ini the ozone distribution tar files if you are interested). It was not mentioned exactly what should be discussed (I guess not the checkstyle changes or the scripts), but let me repeat here: This patch is not about removing yetus support or remove anything from the existing build process. It provides additional checks which may or may not useful. (They are more strict checks she code can be better with these checks) I would also prefer to discuss about the goals regarding to the ozone builds. Our current problems are: * We would like to run the docker based acceptance tests for each patch (dnd is required) * We would like to run full unit test / checks for all the hadoop-hdds/hadoop-ozone project for all the patch * The unit tests from hadoop-ozone/integration-test should be executed for all the patches * Some of the existing checks (for example shaded client) can be ignore for hdds/ozone only patches * I agreed with AW that it would be better to strictly avoid patches which modifies both ozone/hdds and other hadoop subprojects in the same time. * Ozone doesn't require the build of Yarn/mapreduce components (handled with -pl + -am maven flags as of now) * And I would prefer to do it at least as fast as the local build (2-3 minutes ozone build as of now) but keep all the required safety options. I think some of these (all?) can be implemented in yetus, but I am not sure how. I would really appreciate any help here. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738168#comment-16738168 ] Anu Engineer commented on HDDS-965: --- [~busbey] I don't know if you are aware of this, originally we had most of the changes done as part of Hadoop. Then during merge we had to spend lots of time to separate the changes out to make sure that Ozone changes will not impact current Hadoop, which I guess was the largest concern when Ozone was merged. One of those JIRAs is HDFS-13258, there are many more if you are interested. I am hopeful that most of the improvements that we have made in ozone can be folded back into Hadoop at some point of time. I would be happy to walk over the list of improvements that I think we should take back into Hadoop. If you would like to champion them, I am willing to do the work of moving them back. I personally believe keeping Ozone under a separate flag (-Phdds), and forcing the community to learn to compile Ozone is more egregious than having a different test system. Hopefully, we can address some of these issues and make Ozone integrate more fundamentally to Hadoop, than being an overlay. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737727#comment-16737727 ] Sean Busbey commented on HDDS-965: -- If we're one project, we should have a common methodology to things like tests so that folks don't need to learn and track different ones. If Ozone's future is an independent project then we should be aimed there from the start, e.g. with its own repo and treated as a podling if not outright in the incubator. I'll go back and re-read the original discussion and community promises, then start a DISCUSS thread since I think this needs more project visibility then it's going to get on some random build quality improvement JIRA (as I am painfully aware of how few other folks track those). > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737670#comment-16737670 ] Anu Engineer commented on HDDS-965: --- [~busbey] if this is a feature that is interesting and would be a good candidate to move to Hadoop/Common, I am very much open to that. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737669#comment-16737669 ] Anu Engineer commented on HDDS-965: --- [~busbey] Thanks for your comment. The way ozone operates is by building on top of Hadoop Infrastructure. The reason is that when merged one of the concerns was that Ozone's changes should not impact Hadoop or Current HDFS in anyway. We have kept up with that philosophy. I personally believe there are several things in Ozone that would benefit HDFS and Common, and I would love to move them back into Hadoop. At this point of time, we are just making sure that we maintain the promises that were made to the community; but I am open to a discussion and cherry-picking any feature that you would like to backport into Hadoop. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737661#comment-16737661 ] Sean Busbey commented on HDDS-965: -- Can you point me to a place to discuss this? It seems a waste to bifrucate our efforts this way. If Ozone is a part of the Hadoop project, we should keep our test efforts consolidated. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737591#comment-16737591 ] Hudson commented on HDDS-965: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15747 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15747/]) HDDS-965. Ozone: checkstyle improvements and code quality scripts. (elek: rev 32cf0412ce4c982ac9ce3e0e9c5812664de78f1b) * (edit) hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java * (edit) hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java * (add) hadoop-ozone/Jenkinsfile * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/client/TestVolume.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/PendingDeleteHandler.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/ozShell/TestOzoneShell.java * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/VolumeManager.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/TestUtils.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerInfo.java * (edit) hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/RandomKeyGenerator.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/GrpcReplicationService.java * (add) hadoop-ozone/dev-support/checks/rat.sh * (edit) hadoop-ozone/objectstore-service/src/main/java/org/apache/hadoop/ozone/web/interfaces/Bucket.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2PipelineMap.java * (add) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/package-info.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/states/Node2ObjectsMap.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketArgs.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java * (edit) hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyOutputStream.java * (add) hadoop-ozone/dev-support/checks/findbugs.sh * (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientRatis.java * (edit) hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/SCMTestUtils.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestNode2PipelineMap.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/HddsConfServlet.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/codec/OmVolumeArgsCodec.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/web/TestOzoneRestWithMiniCluster.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/OzoneTestUtils.java * (add) hadoop-ozone/dev-support/checks/unit.sh * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java * (edit) hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/UnknownPipelineStateException.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SimpleContainerDownloader.java * (edit) hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/util/OzoneVersionInfo.java * (add) hadoop-ozone/dev-support/checks/checkstyle.sh * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java * (add) hadoop-ozone/dev-support/docker/Dockerfile * (edit) hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/BucketManager.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java * (edit) hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/BlockOutputStream.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/TestSCMContainerPlacementCapacity.java * (edit) hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/genesis/BenchMarkRocksDbStore.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/command/TestCommandStatusReportHandler.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java * (edit) hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyArgs.java * (add) hadoop-ozone/dev-support/checks/build.sh * (edit) hadoop-ozone/c
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737574#comment-16737574 ] Elek, Marton commented on HDDS-965: --- Thanks [~anu] the review. I committed it to the trunk. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737331#comment-16737331 ] Anu Engineer commented on HDDS-965: --- +1, Thanks for taking care of this. Please go ahead and commit. A special thank you for all those scripts that make it easy to run a specific task in the build, That will save a lot of time. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737321#comment-16737321 ] Elek, Marton commented on HDDS-965: --- BTW the Jenkinsfile is in a subdirectory so it's 100% compatible with the existing Yetus build or any Jenkinsfile based Yetus build in the future. The full and more strict tests are executed by the separated jenkins job in addition to any other check. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-965) Ozone: checkstyle improvements and code quality scripts
[ https://issues.apache.org/jira/browse/HDDS-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737315#comment-16737315 ] Elek, Marton commented on HDDS-965: --- The patch fixes about 150 checkstyle issues. With the patch there is no more checkstyle issue in hadoop-ozone/hadoop-hdds projects. It also fixes the remaining ~10 findbugs issues. It also adds some very simple scripts to build/check the project. I think it's more easy to run all the required checks locally with the simplified scripts. For example: hadoop-ozone/dev-support/checks/build.sh does a build (and we don't need to explain all the required maven parameters for a new contributor) hadoop-ozone/dev-support/checks/checkstyle.sh checks the checkstyle issues. The last file in the patch is a Jenkinsfile which can be used to call all the simple check scripts. With the help of the Jenkinsfile it's easy to process all the PR on github. The example build is created on the apache jenkins: [https://builds.apache.org/view/O/view/Ozone%20(Hadoop%20Ozone)/job/Ozone] The build itself is executed in a docker. After HDDS-851 the docker file will be moved to the separated repository and to the apache/ docker namespace. As of now it's uploaded to the docker hub manually. The build image itself contains pre-cached 3rd party maven artifacts (about 1GB). It makes the build extremly fast (25 minutes vs 2 minutes for one stage). Even if the maven repo is separated for all the build the 3rd party jar files are already downloaded to the docker image. Follow-up TODOs: * The detailed status lines are not yet updated in the github pr (see this example about the goal: [https://github.com/elek/prcitest/pull/6)] * The dockerfile will be moved to apache/ozone-build * Acceptance test file exists but the results are not processed, yet. It should be adjusted, but with this approach it will be easy to run for all the PR builds. > Ozone: checkstyle improvements and code quality scripts > --- > > Key: HDDS-965 > URL: https://issues.apache.org/jira/browse/HDDS-965 > Project: Hadoop Distributed Data Store > Issue Type: New Feature >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Experimental scripts to test github pr capabilities after the github url > move. The provided scripts are easier to use locally and provides more > strict/focused checks then the existing pre-commit scripts. But this is not a > replacements of the existing yetus build as it adds additional (more strict) > checks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
