> On 22 Sep 2019, at 09:33, Barak Korren <[email protected]> wrote: > > > > On Thu, 19 Sep 2019 at 17:13, Ehud Yonasi <[email protected] > <mailto:[email protected]>> wrote: > Hey everyone, > Following the presentation we did last week [1] > <https://bluejeans.com/s/zAjyX/>, I wanted to summarize the new patch gating > workflow that will be pushed to oVirt soon and will impact all the developers. > > Summary: > > The purpose of the new workflow is to verify patches earlier ( shift left ) > before they are merged and provide much faster feedback for developers if > their patch fails OST. > > Feedback from OST will now be posted directly to Gerrit instead of requiring > human intervention from the infra team to notify developers > > We expect developers to check why their patch is not passing the gate (OST), > debug it, find the root cause and fix it before merging the patch. > > Any concerns regarding the stability of OST should be communicated and > addressed ASAP. > The status today is that if OST fails post-merge gating packages are not > pushed to tested and QE doesn’t get to test them. The change with pre-merge > gating is that patches won’t be merged if OST fails, so if there are any > fragile or flaky tests they should be examined by their maintainers and > fixed, skipped or removed by the maintainers. > > FYI, we are not removing the Merge button at this point, so maintainers will > still be able to merge patches that they believe 100% is not breaking the > build and failing OST tests. > > Please note that merging patches that break OST will cause it to start > failing for all other patches, we urge you to avoid trying to bypass it if at > all possible. > > In the following section, I will explain more on Patch Gating, how to onboard > it, etc. > > > FAQ on oVirt’s Gating System and how to onboard your project on it: > > Q. What is Patch Gating? > A. It is triggered pre-merge on patches and running OST as the gate system > tests, unlike today > where we have post-merge OST that runs the patches after the projects are > merged. This means developers get early feedback on their patches if it is > passing OST. > > Q. What causes the gating process to start? > A. Once a patch is verified, passed CI and has Code-Review +2 labels, the > gating process will be started. You will receive a message in the patch > > Q. How does it report results to my patches? > A. A comment will be posted in your patch with the job URL failure. > > > Q. How will my patch get merged? > A. If the patch has passed the gating (OST), Zuul (The new CI system for > patch gating) will merge the patch automatically. > > > Q. How do I onboard my project? > A. > Open a JIRA ticket or mail to [email protected] > <mailto:[email protected]> > Creating a file named 'zuul.yaml' under your project root OR > `zuul.d/zuul.yaml` and fill with the following content: > > - project: > templates: > - ost-gated-project > > > Q. My projects run on STDCI V1, is that ok? > A. No, the patch gating logic runs on STDCI V2 only! meaning that you will > have to shift your project to V2. > If you need help regarding the transition to V2 you can open a JIRA[2] > <https://ovirt-jira.atlassian.net/>ticket or mail to [email protected] > <mailto:[email protected]> > and visit the docs [3] <https://ovirt-infra-docs.readthedocs.io/en/latest/>. > > Q. What if I want to merge the patch regardless of OST results? > A. If you are a maintainer of the project, you can still merge it. we are not > removing the merge button option. > But, merging when failing OST can break your project so merging on failure is > unadvertised. > > Q. What if my patch failing because of dependency on different project patch? > A. Patch Gating (Zuul) has a mechanism for cross-project dependency! All you > need to do is to add to the > commit message the patch URL you are dependent on: > > Depends-On: https://gerrit.ovirt.org/patch_number > <https://gerrit.ovirt.org/patch_number> > And they will be tested together. > > Note: you can have multiple dependencies. > > Q. How do I debug OST? > A. There are various ways of looking in the logs and output for the errors: > Blue Ocean view, > <https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_gate/> > you can see the jobs that were run inside the gate and find the suites which > failed. > ci_build_summary view, An internal tool to view the threads and redirect to > the specific logs/artifacts. > Test results analyzer, available if the tests were run. you can view the > failed tests and their output and OST maintainers and your team leads should > be able to assist. > For further learning on how to debug OST please visit the OST FAQ > <https://drive.google.com/open?id=1Sohq7bdgZS341gs5-lvB9lyS0GXawqaIvuUziOWwoGQ>. > > > Q. Will the current infrastructure be able to support all the patches? > A. The CI team has made tremendous work in utilizing the infrastructure. > The OST gating will run inside OpenShift pods unlike before as bare metals > and we can > gain from that right now approximately 50 pods in parallel to run > simultaneously and we will review adding more if the need arises. > > Q. When I have multiple patches, in which order will they be tested by the > gating system? > A. The patches will be tested as the flow they will be merged. The gating > system knows how to simulate patches post-merge. > > Q. What do I do if I think OST failed because of an infra issue and not my > patch? > A. You can contact the CI team by sending mail to [email protected] > <mailto:[email protected]> and explain your concerns + sending the > patch URL. > > Q. Will check-merged scripts be used by the gating system? > A. No, they will be used in the current workflow with OST post-merge gating > system called Change-Queue. > > Q. Can I add my own tests to the gating system? > A. The gating system is running OST tests, so if it’s a test that should be > included in the OST, then yes. > > Q. What will happen to the old change-queue system now that we have gating? > A. At this time, the change-queue system will stay and gate post-merge jobs > until all of oVirt projects will onboard to patch gating. > We might consider using the change-queue for further coverage of tests in the > future. > > Q. How can I re-trigger a failed patch to the gate again? > A. There are 2 options to retrigger: > If the case is to fix your patch, just uploaded a new patchset and turn the > Code-Review, Verified and CI labels again. > If you want to re-trigger the same patchset again just write a comment in > Gerrit: > ‘ci emulate-gate please’ > > > The command here is "ci gate please", the "emulate-gate" command is for doing > a gate dry-run for debugging the gate itself. It should not be used by oVirt > developers.
can we please get a short comment posted by the bot describing a tl;dr version of commands one can run on the patch? It would be extremely helpful! > > > > > Q. I usually write a series of related patches that should be merged > together, can the Gating system test all of them in a single test? > A. No, they will be tested in parallel as the number of patches in the > series. This is why we’ve increased our capacity to run OST for this case. > > > Architectural design document [4] > <https://drive.google.com/open?id=1qV_iNJL6jHARlti7zpnRZRfQM_Q9Z-w8ONvcKEmaAgA> > would provide you the understanding of the patch gating process with the new > services we will be using. > > > > [1]: https://bluejeans.com/s/zAjyX/ <https://bluejeans.com/s/zAjyX/> > [2]: https://ovirt-jira.atlassian.net <https://ovirt-jira.atlassian.net/> > [3]: https://ovirt-infra-docs.readthedocs.io/en/latest/ > <https://ovirt-infra-docs.readthedocs.io/en/latest/> > [4] > https://drive.google.com/open?id=1qV_iNJL6jHARlti7zpnRZRfQM_Q9Z-w8ONvcKEmaAgA > <https://drive.google.com/open?id=1qV_iNJL6jHARlti7zpnRZRfQM_Q9Z-w8ONvcKEmaAgA> > > _______________________________________________ > Devel mailing list -- [email protected] <mailto:[email protected]> > To unsubscribe send an email to [email protected] > <mailto:[email protected]> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/> > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/KKEHCJHRS645H4LNDPB4PKRBUBZEYLOY/ > > <https://lists.ovirt.org/archives/list/[email protected]/message/KKEHCJHRS645H4LNDPB4PKRBUBZEYLOY/> > > > -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com <http://redhat.com/> | TRIED. TESTED. TRUSTED. | > redhat.com/trusted > <http://redhat.com/trusted>_______________________________________________ > Devel mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/557ULYA4WT6ZW7327W2XSPVRICGUZTU4/
_______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/P4AZE33FVYLNKXNVHYTWC4ZIY5DUYKUI/
