On Mon, Nov 27, 2023 at 9:34 PM <num...@ovn.org> wrote: > > From: Numan Siddique <num...@ovn.org> > > This patch series adds incremental processing in the lflow engine > node to handle changes to northd and other engine nodes. > Changed related to load balancers and NAT are mainly handled in > this patch series. > > This patch series can also be found here - > https://github.com/numansiddique/ovn/tree/northd_lflow_ip/v1
Correction - https://github.com/numansiddique/ovn/tree/northd_lbnatacl_lflow/v3 Thanks Numan > > Prior to this patch series, most of the changes to northd engine > resulted in full recomputation of logical flows. This series > aims to improve the performance of ovn-northd by adding the I-P > support. In order to add this support, some of the northd engine > node data (from struct ovn_datapath) is split and moved over to > new engine nodes - mainly related to load balancers, NAT and ACLs. > > Below are the scale testing results done with these patches applied > using ovn-heater. The test ran the scenario - > ocp-500-density-heavy.yml [1]. > > With all the lflow I-P patches applied, the resuts are: > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Min (s) Median (s) 90%ile (s) > 99%ile (s) Max (s) Mean (s) Total (s) Count Failed > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Iteration Total 0.136883 1.129016 1.192001 > 1.204167 1.212728 0.665017 83.127099 125 0 > Namespace.add_ports 0.005216 0.005736 0.007034 > 0.015486 0.018978 0.006211 0.776373 125 0 > WorkerNode.bind_port 0.035030 0.046082 0.052469 > 0.058293 0.060311 0.045973 11.493259 250 0 > WorkerNode.ping_port 0.005057 0.006727 1.047692 > 1.069253 1.071336 0.266896 66.724094 250 0 > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > The results with the present main are: > > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Min (s) Median (s) 90%ile (s) > 99%ile (s) Max (s) Mean (s) Total (s) Count Failed > ------------------------------------------------------------------------------------------------------------------------------------------------------- > Iteration Total 0.135491 2.223805 3.311270 > 3.339078 3.345346 1.729172 216.146495 125 0 > Namespace.add_ports 0.005380 0.005744 0.006819 > 0.018773 0.020800 0.006292 0.786532 125 0 > WorkerNode.bind_port 0.034179 0.046055 0.053488 > 0.058801 0.071043 0.046117 11.529311 250 0 > WorkerNode.ping_port 0.004956 0.006952 3.086952 > 3.191743 3.192807 0.791544 197.886026 250 0 > ------------------------------------------------------------------------------------------------------------------------------------------------------- > > [1] - > https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml > > > Note: This cover letter will be enhanced with more details in v4. > > v2 -> v3 > ------- > * Addressed some of the review comments from Han and Dumitru. There > are still a few pending review comments which needs to be addressed > or discussed. > > * Renamed the engine node from "lr_lbnat_data" to "lr_stateful" > (v3 patch 5). > > * Renamed the engine node from "ls_lbacls" to "ls_stateful" (v3 patch 8). > > * Removed v2 patch 2 from the series (northd: Track ovn_datapaths in > northd engine track data."). This patch is now part of v3 patch 7 > (northd: Add a new node 'ls_stateful'). > > * Squashed v2 patch 8 (northd: Don't commit dhcp response flows in > the conntrack.) into v3 patch 7 (northd: Add a new node > 'ls_stateful'.) > > > v1 -> v2 > -------- > * Now also maintaing array indexes for ls_lbacls, lr_nat and > lr_lb_nat_data tables (similar to ovn_datapaths->array) to > make the lookup effecient. The same ovn_datapath->index > is reused. > > * Made some signficant changes to 'struct lflow_ref' in lflow-mgr.c. > In v2 we don't use objdep_mgr to maintain the resource to lflow > references. Instead we maintain the 'struct lflow' pointer. > With this we don't need to maintain additional hmap of lflows. > > > > Numan Siddique (16): > northd: Refactor the northd change tracking. > tests: Add a couple of tests in ovn-northd for I-P. > northd: Move router ports SB PB options sync to sync_to_sb_pb node. > northd: Add a new engine 'lr_nat' to manage lr NAT data. > northd: Add a new engine 'lr_stateful' to manage lr's stateful data. > northd: Generate router's stateful flows using lr_stateful data. > northd: Add a new node 'ls_stateful'. > northd: Refactor lflow management into a separate module. > northd: Use lflow_ref when adding all logical flows. > northd: Move ovn_lb_datapaths from lib to northd module. > northd: Handle lb changes in lflow engine. > northd: Add lr_stateful handler for lflow engine node. > northd: Add ls_stateful handler for lflow engine node. > northd: Add I-P for NB_Global and SB_Global. > northd: Add a noop handler for northd SB mac binding. > northd: Add northd change handler for sync_to_sb_lb node. > > lib/lb.c | 96 - > lib/lb.h | 57 - > lib/ovn-util.c | 17 +- > lib/ovn-util.h | 2 +- > lib/stopwatch-names.h | 3 + > northd/aging.c | 21 +- > northd/automake.mk | 12 +- > northd/en-global-config.c | 588 ++++ > northd/en-global-config.h | 65 + > northd/en-lflow.c | 122 +- > northd/en-lflow.h | 8 + > northd/en-lr-nat.c | 423 +++ > northd/en-lr-nat.h | 130 + > northd/en-lr-stateful.c | 678 ++++ > northd/en-lr-stateful.h | 136 + > northd/en-ls-stateful.c | 448 +++ > northd/en-ls-stateful.h | 104 + > northd/en-northd.c | 70 +- > northd/en-northd.h | 2 +- > northd/en-port-group.h | 3 + > northd/en-sync-from-sb.c | 2 +- > northd/en-sync-sb.c | 510 ++- > northd/inc-proc-northd.c | 71 +- > northd/lflow-mgr.c | 1078 ++++++ > northd/lflow-mgr.h | 192 ++ > northd/northd.c | 6510 ++++++++++++++++--------------------- > northd/northd.h | 593 +++- > northd/ovn-northd.c | 7 + > tests/ovn-macros.at | 44 + > tests/ovn-northd.at | 953 +++++- > 30 files changed, 8825 insertions(+), 4120 deletions(-) > create mode 100644 northd/en-global-config.c > create mode 100644 northd/en-global-config.h > create mode 100644 northd/en-lr-nat.c > create mode 100644 northd/en-lr-nat.h > create mode 100644 northd/en-lr-stateful.c > create mode 100644 northd/en-lr-stateful.h > create mode 100644 northd/en-ls-stateful.c > create mode 100644 northd/en-ls-stateful.h > create mode 100644 northd/lflow-mgr.c > create mode 100644 northd/lflow-mgr.h > > -- > 2.41.0 > > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
