Hi,
Dumitru mentioned to me that the impact of change tracking may affect
the performance relative to the current code. I did some quick tests
with a large NBDB and checked northd timings. The results are below. It
doesn't seem to have a significant impact but the data is a bit noisy.
Without this patch:
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test1 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 2:
ovn-northd delay before processing: 7000ms
ovn-northd completion: 14935ms
Statistics for 'ovnnb_db_run'
Total samples: 5
Maximum: 7915 msec
Minimum: 0 msec
95th percentile: 0.000000 msec
Short term average: 6732.375000 msec
Long term average: 7831.435378 msec
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test2 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 3:
ovn-northd delay before processing: 6645ms
ovn-northd completion: 14365ms
Statistics for 'ovnnb_db_run'
Total samples: 5
Maximum: 7600 msec
Minimum: 0 msec
95th percentile: 0.000000 msec
Short term average: 6618.250000 msec
Long term average: 7460.322269 msec
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test3 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 4:
ovn-northd delay before processing: 6670ms
ovn-northd completion: 14362ms
Statistics for 'ovnnb_db_run'
Total samples: 5
Maximum: 7568 msec
Minimum: 0 msec
95th percentile: 0.000000 msec
Short term average: 6615.750000 msec
Long term average: 7476.213026 msec
With this patch:
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test1 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 2:
ovn-northd delay before processing: 112ms
ovn-northd completion: 7986ms
Statistics for 'ovnnb_db_run'
Total samples: 3
Maximum: 7732 msec
Minimum: 7535 msec
95th percentile: 7535.000000 msec
Short term average: 7620.500000 msec
Long term average: 7729.515200 msec
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test2 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 3:
ovn-northd delay before processing: 118ms
ovn-northd completion: 7826ms
Statistics for 'ovnnb_db_run'
Total samples: 3
Maximum: 7564 msec
Minimum: 7562 msec
95th percentile: 7562.000000 msec
Short term average: 7563.250000 msec
Long term average: 7563.000100 msec
ovn-appctl -t ovn-northd stopwatch/reset ovnnb_db_run && ovn-nbctl
--print-wait-time --wait=sb lsp-add lswitch-ovn-scale-77 test3 &&
ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
Time spent on processing nb_cfg 4:
ovn-northd delay before processing: 109ms
ovn-northd completion: 7880ms
Statistics for 'ovnnb_db_run'
Total samples: 3
Maximum: 7767 msec
Minimum: 7555 msec
95th percentile: 7555.000000 msec
Short term average: 7623.000000 msec
Long term average: 7615.904800 msec
On 03/09/2021 13:21, Mark Gray wrote:
> Part 1:
> The first two comits deal with reorganising the northd code base a little.
> Please note that this commit mainly involves moving code around with minimal
> code changes. However, due to tight coupling between ovn-northd.c and
> northd.c,
> some minor changes were needed. For reference, and to help reviews, please
> examine the following at a minimum:
>
> * Configuration of the probe interval in northd.c
> (ovsdb_idl_set_probe_interval())
> * Passing of "use_parallel_build" and "lflow_locks" from ovn-northd.c and
> northd.c.
> * Update of "struct northd_context": additon of fields and move to h file.
>
> The commits were (hopefully) structured in a way to make the review easier. As
> this change touches all of ovn-northd, any change to "master" will make a
> rebase
> necessary and probably difficult. Therefore, if the general ideas is OK, then
> it would be great if this series could be expedited to prevent many rebases!
>
> Part 2:
> The remaining commits add the inc-proc-eng framework to northd. This does
> *not*
> add any incremental processing of northd processing at this stage but
> provides the
> framework. Even in this base configuration, we see an advantage as northd no
> longer
> processes the databases if it has been woken up only to handle, for example, a
> unixctl command. This can be seen below:
>
> $ ovn-appctl -t ovn-northd stopwatch/reset
> $ for i in {1..10}; do ovn-appctl -t ovn-northd stopwatch/show >/dev/null;
> done
> $ ovn-appctl -t ovn-northd stopwatch/show ovnnb_db_run
> Statistics for 'ovnnb_db_run'
> Total samples: 0
> Maximum: 0 msec
> Minimum: 0 msec
> 95th percentile: 0.000000 msec
> Short term average: 0.000000 msec
> Long term average: 0.000000 msec
>
> Hopefully this starting point will allow others to discuss or contribute
> changes to
> incrementally process some aspects of northd. We can also decide if it is
> worth
> progressing with "Part 2" if we think there are potential benefits until we
> start
> seeing the benefits of DDlog.
>
> Thanks
>
> ---
> v2: Rebase
> v3: Rebase. Fixed compile-time error
> v4: Add additional commits which add framework for incremental processing in
> northd
>
> Mark Gray (7):
> ovn-northd: Rename ovn-northd.c to northd.c
> northd: Split northd.c
> inc-proc-eng: Allow definition of engine_node with global scope
> northd: Introduce incremental processing for northd
> northd: Add n_nat_entries field to 'struct ovn_datapath'
> northd: Add runtime node
> northd: Add functionality to runtime node
>
> Documentation/tutorials/ovn-openstack.rst | 154 +-
> controller/ovn-controller.c | 2 +-
> lib/inc-proc-eng.h | 24 +-
> northd/automake.mk | 8 +
> northd/en-northd.c | 52 +
> northd/en-northd.h | 17 +
> northd/en-runtime.c | 66 +
> northd/en-runtime.h | 25 +
> northd/inc-proc-northd.c | 258 +
> northd/inc-proc-northd.h | 15 +
> northd/lrouter.dl | 2 +-
> northd/northd.c | 14568 +++++++++++++++++++
> northd/northd.h | 45 +
> northd/ovn-northd.c | 15084 +-------------------
> northd/ovn.rs | 2 +-
> northd/ovn_northd.dl | 2 +-
> tests/ovn-northd.at | 2 +-
> 17 files changed, 15454 insertions(+), 14872 deletions(-)
> create mode 100644 northd/en-northd.c
> create mode 100644 northd/en-northd.h
> create mode 100644 northd/en-runtime.c
> create mode 100644 northd/en-runtime.h
> create mode 100644 northd/inc-proc-northd.c
> create mode 100644 northd/inc-proc-northd.h
> create mode 100644 northd/northd.c
> create mode 100644 northd/northd.h
>
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
