On 6/30/21 2:49 PM, Mark Gray wrote: > On 24/06/2021 16:34, Dumitru Ceara wrote: >> On 6/18/21 10:52 AM, Mark Gray wrote: >>> Add a suite of micro-benchmarks to aid a developer in understanding the >>> performance impact of any changes that they are making. They can be used to >>> help to understand the relative performance between two test runs on the >>> same >>> test machine, but are not intended to give the absolute performance of OVN. >>> >>> To invoke the performance testsuite, run: >>> >>> $ make check-perf >>> >>> This will run all available performance tests. >>> >>> Additional metrics (e.g. memory, coverage, perf counters) may be added >>> in the future. Additional tests (e.g. additional topologies, ovn-controller >>> tests) may be added in the future. >>> >>> Signed-off-by: Mark Gray <mark.d.g...@redhat.com> >>> --- >> >> Thanks for this, I think this is a very good idea! > > Thanks (and thanks for the review!), I think it is a good starting part > but it will only be useful if it gets used! Personally, I can see myself > using this. However, if it doesn't, it is not very invasive and could > easily be let rot or removed. >> >>> >>> Notes: >>> v2: create results directory to fix build error >>> v3: forgot to commit, create results directory to fix build error >>> >>> Documentation/topics/testing.rst | 49 ++++++++ >>> tests/.gitignore | 3 + >>> tests/automake.mk | 27 ++++ >>> tests/perf-northd.at | 207 +++++++++++++++++++++++++++++++ >>> tests/perf-testsuite.at | 26 ++++ >>> 5 files changed, 312 insertions(+) >>> create mode 100644 tests/perf-northd.at >>> create mode 100644 tests/perf-testsuite.at >>> >>> diff --git a/Documentation/topics/testing.rst >>> b/Documentation/topics/testing.rst >>> index be9e7c57331c..ccd3278437b1 100644 >>> --- a/Documentation/topics/testing.rst >>> +++ b/Documentation/topics/testing.rst >>> @@ -256,3 +256,52 @@ the following:: >>> All the features documented under `Unit Tests`_ are available for the >>> datapath testsuites, except that the datapath testsuites do not >>> support running tests in parallel. >>> + >>> +Performance testing >>> +~~~~~~~~~~~~~~~~~~~ >>> + >>> +OVN includes a suite of micro-benchmarks to aid a developer in >>> understanding the >>> +performance impact of any changes that they are making. They can be used to >>> +help to understand the relative performance between two test runs on the >>> same >>> +test machine, but are not intended to give the absolute performance of OVN. >>> + >>> +To invoke the performance testsuite, run:: >>> + >>> + $ make check-perf >>> + >>> +This will run all available performance tests. Some of these tests may be >>> +long-running as they need to build complex logical network topologies. In >>> order >>> +to speed up subsequent test runs, some objects (e.g. the Northbound DB) >>> may be >>> +cached. In order to force the tests to rebuild all these objects, run:: >>> + >>> + $ make check-perf TESTSUITEFLAGS="--rebuild" >>> + >>> +A typical workflow for a developer trying to improve the performance of OVN >>> +would be the following: >>> + >>> +0. Optional: Modify/add a performance test to buld the topology that you >>> are >>> + benchmarking, if required. >>> +1. Run ``make check-perf TESTSUITEFLAGS="--rebuild"`` to generate cached >>> + databases. >>> + >>> +.. note:: >>> + This step may take some time depending on the number of tests that are >>> being >>> + rebuilt, the complexity of the tests and the performance of the test >>> machine. >>> + If you are only using one test, you can specify the test to run by >>> adding the >>> + test number to the ``make`` command. >>> + (e.g. ``make check-perf TESTSUITEFLAGS="--rebuild <test number>"``) >>> + >>> +2. Run ``make check-perf`` to measure the performance metric that you are >>> + benchmarking against. If you are only using one test, you can specify >>> the test to run by adding the test number to the ``make`` command. >>> + (e.g. ``make check-perf TESTSUITEFLAGS="--rebuild <test number>"``) >> >> It's not very clear where the user would find the test results: >> >> tests/perf-testsuite.dir/results > > It is mentioned further below but I will move it up to here to make it > clearer.
Ah, OK, I missed it completely. [...] >> >> The file is called perf-northd.at so if we ever decide to add some tests >> to measure end-to-end performance including ovn-controller I assume >> we'll need a new file, e.g., perf-controller.at. Shouldn't we already >> move all of the generic macros above to a shared file; maybe ovn-macros.at? >> > > This is exactly what I was thinking but I didn't want to do it yet > because a) I am not sure if it will be used much b) I am unsure what > will be common yet. > > As such, I would prefer to hold off but I am happy to do it now if you > think there is benefit. > Ok, makes sense, let's keep it like this and see where it goes. Thanks! _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
