Two big changes are announced here, and the first pertains to allowing other developers and labs to report patch status in the ozlabs patchwork UI to assist maintainers making decisions on patch inclusion.
To make this as easy as possible, the robot will take a known form of email sent to the ovs-bu...@openvswitch.org mailing list and push the information into patchwork. This information can be browsed either via the web UI or programmatically via the checks API. The second change is to de-emphasize the use of Travis-CI as a project wide resource. Travis-CI might still be useful for individual developers testing small patches, but can no longer fulfill the role of project-wide continuous integration service. Remove the badge from the front page, and note the other CI services that OVS supports. Rather than completely drop all references to travis, simply emphasize using GitHub Actions as the preferred CI solution. Signed-off-by: Aaron Conole <acon...@redhat.com> Refs: https://mail.openvswitch.org/pipermail/ovs-dev/2021-June/384040.html --- .../contributing/submitting-patches.rst | 8 +- Documentation/topics/testing.rst | 122 ++++++++++++------ NEWS | 3 + README.rst | 2 - 4 files changed, 90 insertions(+), 45 deletions(-) diff --git a/Documentation/internals/contributing/submitting-patches.rst b/Documentation/internals/contributing/submitting-patches.rst index 4a6780371d..a6a10f8d89 100644 --- a/Documentation/internals/contributing/submitting-patches.rst +++ b/Documentation/internals/contributing/submitting-patches.rst @@ -68,10 +68,10 @@ Testing is also important: feature. A bug fix patch should preferably add a test that would fail if the bug recurs. -If you are using GitHub, then you may utilize the travis-ci.org and the GitHub -Actions CI build systems. They will run some of the above tests automatically -when you push changes to your repository. See the "Continuous Integration with -Travis-CI" in :doc:`/topics/testing` for details on how to set it up. +If you are using GitHub, then you may utilize the the GitHub Actions CI build +systems. They will run some of the above tests automatically when you push +changes to your repository. See the "Continuous Integration" section in +:doc:`/topics/testing` for details on continuous integration. Email Subject ------------- diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index 951fe9e851..7e0ced8855 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -416,45 +416,89 @@ You should invoke scan-view to view analysis results. The last line of output from ``clang-analyze`` will list the command (containing results directory) that you should invoke to view the results on a browser. -Continuous Integration with Travis CI -------------------------------------- - -A .travis.yml file is provided to automatically build Open vSwitch with various -build configurations and run the testsuite using Travis CI. Builds will be -performed with gcc, sparse and clang with the -Werror compiler flag included, -therefore the build will fail if a new warning has been introduced. - -The CI build is triggered via git push (regardless of the specific branch) or -pull request against any Open vSwitch GitHub repository that is linked to -travis-ci. - -Instructions to setup travis-ci for your GitHub repository: - -1. Go to https://travis-ci.org/ and sign in using your GitHub ID. -2. Go to the "Repositories" tab and enable the ovs repository. You may disable - builds for pushes or pull requests. -3. In order to avoid forks sending build failures to the upstream mailing list, - the notification email recipient is encrypted. If you want to receive email - notification for build failures, replace the encrypted string: - - 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis - 2. In your Open vSwitch repository: travis encrypt myl...@mydomain.org - 3. Add/replace the notifications section in .travis.yml and fill in the - secure string as returned by travis encrypt:: - - notifications: - email: - recipients: - - secure: "....." - - .. note:: - You may remove/omit the notifications section to fall back to default - notification behaviour which is to send an email directly to the author and - committer of the failing commit. Note that the email is only sent if the - author/committer have commit rights for the particular GitHub repository. - -4. Pushing a commit to the repository which breaks the build or the - testsuite will now trigger a email sent to myl...@mydomain.org +Continuous Integration +---------------------- + +The Open vSwitch project can make use of multiple public CI services to help +developers ensure their patches don't introduce additional regressions. Each +service requires different forms of configuration, and for the supported +services the configuration file(s) to automatically build Open vSwitch with +various build configurations and run the testsuites is/are available. For +example, the GitHub Actions builds will be performed with gcc, sparse and clang +with the -Werror compiler flag included, therefore the build will fail if a new +warning has been introduced. + +Each ci system uses a different method of enablement, but most are available +from the GitHub settings page. By default, Open vSwitch includes a GitHub +actions running configuration, and this will automatically email the repository +owner. + +Currently, Open vSwitch project enables the following public CI services along +with the appropriate configuration settings:: + + - AppVeyor: appveyor.yml + - Cirrus-CI: .cirrus.yml + - GitHub Actions: .github/workflows + - Travis-CI: .travis.yml + +GitHub Actions is available without any additional configuration. + +Additionally, as each patch is posted to the mailing list, the public CI +machinery will run additional tests on the patches and report results. + +Bring Your Own Lab +------------------ + +The Open vSwitch project makes use of the ozlabs patchwork instance for +patch status and management. This patch tracking tool provides information +on the state of changes proposed for Open vSwitch. One feature that the +patchwork instance offers is for the concept of 'Checks' to be reported. +These are sets of test functionality that can be executed and present +status 'at-a-glance' with the ability to drill deeper into why a particular +test failed. + +To this end, the Open vSwitch "0-day Robot" will accept emails sent to +ovs-bu...@openvswitch.org formatted in the right way to be reflected on this +page. This allows any lab to contribute to the testing and validation of +patches. + +To report a test status to a particular patch, send exactly one email to +the mailing formatted as such:: + + From: your email <addr...@server.com> + To: ovs-bu...@openvswitch.org + Date: Mon, 28 Jun 2021 00:00:00 +0000 + Subject: |STATUS| pwPATCHID commit subject + + Test-Label: your-robot-or-test-name + Test-Status: STATUS + http://patchwork.ozlabs.org/api/patches/PATCHID/ + + _ONE LINE DESCRIPTION_ + + Addtional details + +In the above example, the STATUS is one of "success" "warning" "fail" depending +on the outcome of the test. This value is case-sensitive. PATCHID should be the +patch for which the test was executed, and ONE LINE DESCRIPTION should be a simple +one line result description, ex: "robot-test: success" + +It is strongly recommended that if the patch is reporting a failure case, one +of the patch authors is cc'd as well. + +Example:: + + From: 0-Day Robot <ro...@bytheb.org> + To: ovs-bu...@openvswitch.org + Date: Fri, 25 Jun 2021 14:30:37 +0400 + Subject: |success| pw1497375 [ovs-dev] [PATCH v2] checkpatch: Ignore macro definitions of FOR_EACH + + Test-Label: github-robot + Test-Status: success + http://patchwork.ozlabs.org/api/patches/1497375/ + + _github build: passed_ + Build URL: https://github.com/ovsrobot/ovs/actions/runs/972188336 vsperf ------ diff --git a/NEWS b/NEWS index b9cb74e9e6..a771eec1a1 100644 --- a/NEWS +++ b/NEWS @@ -21,6 +21,9 @@ Post-v2.15.0 - ovsdb-tool: * New option '--election-timer' to the 'create-cluster' command to set the leader election timer during cluster creation. + - Testing: + * Integrate "Bring your own lab" to the build reports page, allowing any + testing lab to submit a test report for a patch. v2.15.0 - 15 Feb 2021 diff --git a/README.rst b/README.rst index 319f705154..8e64d74aee 100644 --- a/README.rst +++ b/README.rst @@ -8,8 +8,6 @@ Open vSwitch .. image:: https://github.com/openvswitch/ovs/workflows/Build%20and%20Test/badge.svg :target: https://github.com/openvswitch/ovs/actions -.. image:: https://travis-ci.org/openvswitch/ovs.png - :target: https://travis-ci.org/openvswitch/ovs .. image:: https://ci.appveyor.com/api/projects/status/github/openvswitch/ovs?branch=master&svg=true&retina=true :target: https://ci.appveyor.com/project/blp/ovs/history .. image:: https://api.cirrus-ci.com/github/openvswitch/ovs.svg -- 2.31.1 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
