Hi Yujun, Thanks for the feedback.
Releng aims to provide a similar service to the OPNFV projects like you explained but some of the limitations you listed below are due to Jenkins itself and how to manage CI in a consistent manner. We need to keep some kind of alignment and ensure that what we are doing on Jenkins and CI in general fits into the overall framework. In general, only the Jenkins job configurations and simple wrapper scripts should be located in releng repo. (excluding common utilities such as Test Result API which will move into a separate repo) The scripts that do project specific stuff such as running unit tests, builds and so on should be owned, developed, and maintained by corresponding projects. The overall structure followed by releng is like this or from [1]. - preprocessing - actual build/test - postprocessing Preprocessing is simply doing general cleanup stuff, fixing permissions and so on before the actual build starts and this is common for all projects. Postprocessing deals with similar common stuff such as uploading artifacts to artifact repo. The scripts that are consumed by releng from projects are the ones that matter and I expect that these change more often than Jenkins jobs or wrappes themselves. Normally you don't need to touch jobs/wrappers if you are not doing a total structure change such as changing the job type from Freestyle to Multijob. These scripts are generally located in <project_repo>/ci/build.sh, <project_repo>/ci/test.sh which get executed by the wrapper scripts which in turn run by Jenkins jobs. When it comes to docker; it is a bit different since the use of docker containers for the test projects started in releng as common/community initiative and the build scripts have been in releng repo developed/maintained by the community at large, not just releng itself. But this is due wanting to have some kind of alignment and historical reasons which can be changed if the projects require different way of doing builds and so on. (I think Storperf is an example to this.) We have plans to look into Zuul v3 and start a PoC but we are waiting for OpenStack to finalize their migration and fix the issues they are experiencing. But we also need to keep in mind that we go further in our CI comparing to OpenStack so it is not just about OpenStack fixing all the issues but also fulfilling OPNFV needs. (one problem we will face is the number/types of plugins we use on our Jenkins and the impacts of having long running tests.) If you can point to specific examples or list additional needs we can take them into our backlog and improve what we do. [1] https://wiki.opnfv.org/download/attachments/11699697/image2017-6-6%2022%3A56%3A30.png?version=1&modificationDate=1496783473000&api=v2 /Fatih From: <infra-wg-boun...@lists.opnfv.org> on behalf of "Yujun Zhang (ZTE)" <zhangyujun+...@gmail.com> Date: Wednesday, 1 November 2017 at 02:19 To: David McBride <dmcbr...@linuxfoundation.org> Cc: "infra...@lists.opnfv.org" <infra...@lists.opnfv.org>, test-wg <test...@lists.opnfv.org>, "opnfv-tech-discuss@lists.opnfv.org" <opnfv-tech-discuss@lists.opnfv.org>, Trevor Cooper <trevor.coo...@intel.com> Subject: Re: [infra-wg] [infra][releng] proposal for RELENG project to take ownership of docker build resources Not sure I fully understand the word "ownership" and the scope of building resources. For sure it is good to maintain the build scripts and global config in releng. But for project specific configuration, it is not so convenient to manage them in a central repository. Sometimes I have to wait for committers of releng to merge my patch which just fixes a small bug of the the build job for my project. Most time the validation of releng job is only possible after the patch is merged. When I found it does not work, I have to go over the steps again. It is not so productive since this workflow involves two teams (releng and project). Ideally, I would expect releng to provide the building resource as a service and the project specific configuration managed inside project repository. This is much like the solution provided by some CI SaaS like Travis-CI[1]. And also the trend in OpenStack infra brought in by zuul v3, which is called in repo configuration. This is not just about docker building, but also applies for other verification job. [1]: https://travis-ci.org/ [2]: https://docs.openstack.org/infra/manual/zuulv3.html#what-is-zuul-v3 On Wed, Nov 1, 2017 at 5:56 AM David McBride <dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org>> wrote: please +1, 0, or -1 to indicate your position Team, This topic came up during our release meeting two weeks ago. I took an action to get a discussion started in the infra meeting. The proposal is for the RELENG project to take ownership of docker build resources (scripts,config files, etc.)<https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/releng> for the purpose of providing a common docker build capability available to all users of OPNFV CI. Active committers to these resources will remain unchanged. We had a good discussion on Monday. There was general consensus that this proposal makes sense. The minutes are here<http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2017/opnfv-meeting.2017-10-30-14.59.html>. I took an action at the infra meeting to start a discussion on the mailing list to seek additional feedback. Please respond to this email to indicate whether you support the proposal. Also feel free to add your comments. Thanks. David -- David McBride Release Manager, OPNFV Mobile: +1.805.276.8018<tel:%2B1.805.276.8018> Email/Google Talk: dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org> Skype: davidjmcbride1 IRC: dmcbride _______________________________________________ infra-wg mailing list infra...@lists.opnfv.org<mailto:infra...@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/infra-wg -- Yujun Zhang
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
