On Sun, Oct 1, 2023 at 9:06 AM Robin Jarry <rja...@redhat.com> wrote: > > Hi Han, > > thanks a lot for your detailed answer. > > Han Zhou, Sep 30, 2023 at 01:03: > > > I think ovn-controller only consumes the logical flows. The chassis and > > > port bindings tables are used by northd to updated these logical flows. > > > > Felix was right. For example, port-binding is firstly a configuration from > > north-bound, but the states such as its physical location (the chassis > > column) are populated by ovn-controller of the owning chassis and consumed > > by other ovn-controllers that are interested in that port-binding. > > I was not aware of this. Thanks. > > > > Exactly, but was the signaling between the nodes ever an issue? > > > > I am not an expert of BGP, but at least for what I am aware of, there are > > scaling issues in things like BGP full mesh signaling, and there are > > solutions such as route reflector (which is again centralized) to solve > > such issues. > > I am not familiar with BGP full mesh signaling. But from what can tell, > it looks like the same concept than the full mesh GENEVE tunnels. Except > that the tunnels are only used when the same logical switch is > implemented between two nodes. > Please note that tunnels are needed not only between nodes related to same logical switches, but also when they are related to different logical switches connected by logical routers (even multiple LR+LS hops away).
> > > So you have enabled monitor_all=true as well? Or did you test at scale > > > with monitor_all=false. > > > > > We do use monitor_all=false, primarily to reduce memory footprint (and also > > CPU cost of IDL processing) on each chassis. There are trade-offs to the SB > > DB server performance: > > > > - On one hand it increases the cost of conditional monitoring, which > > is expensive for sure > > - On the other hand, it reduces the total amount of data for the > > server to propagate to clients > > > > It really depends on your topology for making the choice. If most of the > > nodes would anyway monitor most of the DB data (something similar to a > > full-mesh), it is more reasonable to use monitor_all=true. Otherwise, in > > topology like ovn-kubernetes where each node has its dedicated part of the > > data, or in topologies where you have lots of small "island" such as a > > cloud with many small tenants that never talks to each other, using > > monitor_all=false could make sense (but still need to be carefully > > evaluated and tested for your own use cases). > > I didn't see recent scale testing for openstack, but in past testing we > had to set monitor_all=true because the CPU usage of the SB ovsdb was > a bottleneck. > To clarify a little more, openstack deployment can have different logical topologies. So to evaluate the impact of monitor_all settings there should be different test cases to capture different types of deployment, e.g. full-mesh topology (monitor_all=true is better) v.s. "small islands" toplogy (monitor_all=false is reasonable). > > > The memory usage would be reduced but I don't know to which point. One > > > of the main consumers is the logical flows table which is required > > > everywhere. Unless there is a way to only sync a portion of this table > > > depending on the chassis, disabling monitor_all would save syncing the > > > unneeded tables for ovn-controller: chassis, port bindings, etc. > > > > Probably it wasn't what you meant, but I'd like to clarify that it is not > > about unneeded tables, but unneeded rows in those tables (mainly > > logical_flow and port_binding). > > It indeed syncs only a portion of the tables. It is not depending directly > > on chassis, but depending on what port-bindings are on the chassis and what > > logical connectivity those port-bindings have. So, again, the choice really > > depends on your use cases. > > What about the FDB (mac-port) and MAC binding (ip-mac) tables? I thought > ovn-controller does not need them. If that is the case, I thought that > by default, the whole tables (not only some of their rows) were excluded > from the synchronized data. > FDB and MAC_binding tables are used by ovn-controllers. They are essentially the central storage for MAC tables of the distributed logical switches (FDB) and ARP/Neighbour tables for distributed logical routers (MAC_binding). A record can be populate by one chassis and consumed by many other chassis. monitor_all should work the same way for these tables: if monitor_all = false, only rows related to "local datapaths" should be downloaded to the chassis. However, for FDB table, the condition is not set for now (which may have been a miss in the initial implementation). Perhaps this is not noticed because MAC learning is not a very widely used feature and no scale impact noticed, but I just proposed a patch to enable the conditional monitoring: https://patchwork.ozlabs.org/project/ovn/patch/20231001192658.1012806-1-hz...@ovn.org/ Thanks, Han > Thanks! >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
