Hi Rosemarie,
On 10/23/26 7:15 AM, Rosemarie O'Riorden wrote:
> This commit adds a test that developers can run to check if the changes
> they've made cause a regression in the performance of ovn-controller or
> ovn-northd.
>
> Run from inside the sandbox and compare two commits' runtime and peak
> memory usage for each process.
>
> This test creates a topology consisting of logical switches and routers,
> ACLs, port groups, NAT, DHCP, DNS, QoS, routing policies, and load
> balancers. The user can also provide their own database file for the
> benchmark.
>
> * To run the test with defaults (200 nodes, track northd and controller):
> ./ovn-benchmark.sh
> * To run with 50 nodes: ./ovn-benchmark.sh 50
> * To run with 30 nodes and only track ovn-northd:
> ./ovn-benchmark.sh 30 northd
> * To run with Valgrind Massif tracking allocations:
> ./ovn-benchmark.sh --valgrind
> * To run with a custom db file: ./ovn-benchmark.sh -f file.db
> * To run with debug info printed: ./ovn-benchmark.sh --debug
> * To see detailed usage instructions: ./ovn-benchmark.sh --help
>
> Each "node" is a router-switch pair with associated features:
> - 1 gateway router (with NAT, static routes, routing policies)
> - 1 logical switch (with 9 ports by default, DHCP, DNS, QoS)
> - 5 load balancers (each with 5 backends)
> - ACLs, port groups, and address sets (shared across all nodes)
>
> For example, 200 nodes creates:
> - 200 routers + 200 switches
> - 1,800 logical switch ports (9 per switch)
> - 1,000 load balancers (5 per node)
> - 1,200 NAT rules, 1,200 static routes, 800 routing policies
> - Plus ACLs, DHCP/DNS records, QoS rules, etc.
>
> ovn-benchmark.py creates the topology and ovn-benchmark.sh is the
> wrapper that tracks peak memory and execution time. ovn-benchmark.py was
> built off of ovn-lb-benchmark.py as a starting point.
>
> VmPeak is tracked by default, but Valgrind can be used instead for more
> precise results.
>
> Reported-at: https://redhat.atlassian.net/browse/FDP-978
> Assisted-by: Claude Sonnet 4.5, Claude Code
> Signed-off-by: Rosemarie O'Riorden <[email protected]>
> ---
> BTW:
> A CI component for this patch, which would add a GH action that runs the
> benchmark on the base upstream vs with the patch applied, and compares
> the results to test for regressions, is coming in the future.
>
> v4 -> v5:
> - Incremental engine coverage:
> - Added 21 incremental phases (add/modify/delete) that run after
> topology creation: Each phase operates on ~10% of nodes, and is
> followed by an hv sync. These phases are timed.
> - After each phase, queries inc-engine/show-stats to check whether
> northd or lflow triggered a full recompute. Phases that use NULL
> handlers (routes, policies, DHCP, DNS, QoS, LB disassociation)
> are marked as expected recomputes; unexpected recomputes print a
> WARNING. Failed phases print FAILED. If the results are unknown,
> "?" prints. Otherwise, "OK" prints.
> - Naming:
> - Renamed gateway routers from lr-X to gw-X (ports lr2j/j2lr to
> gw2j/j2gw). Topology diagram updated.
> - Renamed all initial creation functions to use create_* prefix;
> incremental functions use add_/modify_/delete_ prefixes.
> - Timing:
> - Removed per-sub-phase creation timing (DB generation, port
> binding, initial sync) because they don't convey any useful info.
> - Removed DB compaction timing from output (compaction still runs
> for accurate memory readings).
> - Incremental phases show per-phase timing alongside engine stats.
> - Output:
> - Config summary moved from stderr to stdout.
> - Debug prints now show actual IP addresses, MAC addresses, and
> operation details instead of just "Adding NAT rules to router 0".
> - Shell script:
> - Duplicate process detection now reports all duplicate PIDs before
> exiting, not just the first.
> - Engine stats use globbed control socket path
> (sandbox/ovn-northd.*.ctl) instead of hardcoded path.
> - Syntax and validation:
> - Replaced sys.stderr.write()/sys.exit(1) with die() throughout.
> - Added minimum bounds checks (nodes >= 1, ports/vips/backends >= 0).
> - str.removesuffix() replaced with [:-3] for Python 3.7.
> - Port groups dict built once and passed to both consumers.
> - Transaction commit helper accepts UNCHANGED as valid.
> - Guards for ports_per_switch=0 edge case in add/delete LSP phases.
>
> v3 -> v4:
> - Update commit message to reflect # of NAT rules and static routes for IPv6
> - Remove python 3.9 syntax (str.removesuffix()), change out for
> python 3.7 (subnet[:-3])
> - Add timeout for ovn-nbctl --wait=hv sync in time tracking part of
> benchmark, and add warning if timeout is hit.
>
> v2 -> v3:
> - Memory tracking:
> - Dropped the background watcher loop that read RSS periodically.
> Now VmPeak is read once from /proc/<pid>/status instead, so short
> peaks aren't missed.
> - Added --valgrind (-v) to rerun the tracked processes under Massif
> and report peak heap. More accurate but 10-50x slower, so it
> warns above 500 nodes.
> - The processes valgrind hijacks get restarted on exit, Ctrl-C
> included, from their saved cmdline and cwd.
> - Timing:
> - Total time is split into topology generation, OVN sync and db
> compaction, so you can tell which phase regressed.
> - Topology fixes:
> - DHCP options were created but never attached to any port. They're
> assigned now, v4 and v6.
> - Same for the web/db/app address sets -- nothing referenced them, so
> added ACLs that do.
> - Split trusted_networks into v4 and v6 sets; one mixed set can't
> serve both the ip4.dst and ip6.dst matches.
> - Gateway join ports all shared one MAC and 10.0.0.1/8. Each gets
> its own MAC and a real join subnet address now, + routes to match.
> - The port group drop ACL matches outport instead of inport.
> - Chassis names start at 1 to match the sandbox system-id.
> - Safety:
> - A missing router, switch, or port group used to be skipped quietly,
> leaving a half-built topology. These error out now.
> - Errors out on duplicate processe and script running outside the sandbox.
> - Added a SIGINT trap, bounds checks for --backends and --vips, and a
> fix for binding more ports than there are nodes.
> - Docs:
> - testing.rst covers both memory modes and the valgrind tradeoff.
> - Small stuff:
> - Removed an unused json RPC connection.
> - Port group lookups use a dict instead of rescanning the table.
> - Better warnings for misused args, and options that may be confusing.
> - Quoting, output formatting, comments, etc.
>
> v1 -> v2:
> - Topology:
> - Restructured from 1:1 chassis:node to a nodes-per-chassis style
> where each chassis handles a partition or 'batch' of the network (ascii
> diagram added).
> - Added --batch-size (-b) for nodes-per-chassis grouping.
> - Added IPv6 coverage.
> - IP addressing:
> - ip_node()/ip6_node() helpers spread index across two octets to support
> >255 nodes.
> - Added bounds checks for node count and ports-per-switch.
> - Shell fixes:
> - pgrep -x for exact process match.
> - break 2 to exit both watcher loops on process death.
> - trap cleanup EXIT with mktemp -d for temp files.
> - Renamed SECONDS bc it's a bash keyword.
> - python -> python3.
> - Quoted array expansions, fixed PID error messages.
> - Code cleanup:
> - Extracted _add_node(), _add_ports(), _add_acl() helpers.
> - Removed dead code (find_by_name(), seqno).
> - Fixed error variable shadowing imported module 'error'.
> - Docs:
> - Added ovn-benchmark section to testing.rst.
> - Added note in submitting-patches.rst.
> ---
> .../contributing/submitting-patches.rst | 4 +
> Documentation/topics/testing.rst | 39 +
> tutorial/automake.mk | 4 +-
> tutorial/ovn-benchmark.py | 1449 +++++++++++++++++
> tutorial/ovn-benchmark.sh | 561 +++++++
> 5 files changed, 2056 insertions(+), 1 deletion(-)
> create mode 100755 tutorial/ovn-benchmark.py
> create mode 100755 tutorial/ovn-benchmark.sh
>
> diff --git a/Documentation/internals/contributing/submitting-patches.rst
> b/Documentation/internals/contributing/submitting-patches.rst
> index abeb3e9d0..c96fdff84 100644
> --- a/Documentation/internals/contributing/submitting-patches.rst
> +++ b/Documentation/internals/contributing/submitting-patches.rst
> @@ -68,6 +68,10 @@ Testing is also important:
> feature. A bug fix patch should preferably add a test that would
> fail if the bug recurs.
>
> +- To check for memory or performance regressions, you can run
> + ``./ovn-benchmark.sh`` inside the OVN sandbox (``make sandbox``).
> + See the "ovn-benchmark" section of :doc:`/topics/testing` for details.
> +
> If you are using GitHub, then you may utilize the GitHub Actions CI system.
> This will run the above tests automatically when you push changes to your
> repository.
> diff --git a/Documentation/topics/testing.rst
> b/Documentation/topics/testing.rst
> index 951133c29..18508ba62 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -294,6 +294,45 @@ of these cached objects, be sure to rebuild the test.
> The cached objects are stored under the relevant folder in
> ``tests/perf-testsuite.dir/cached``.
>
> +ovn-benchmark
> ++++++++++++++
> +
> +The ``tutorial/`` directory contains a memory and performance benchmarking
> +tool that can be used to detect regressions between commits:
> +
> +- ``ovn-benchmark.sh``: Shell wrapper that tracks peak memory and
> + execution time for ``ovn-northd`` and ``ovn-controller``. By default
> + it reports peak virtual memory (VmPeak) from ``/proc/<pid>/status``,
> + which captures all allocated memory including pages not yet accessed.
> + Pass ``--valgrind`` to restart the tracked processes under Valgrind
> + Massif and report peak heap memory instead; this is more accurate but
> + 10-50x slower.
> +- ``ovn-benchmark.py``: Python script that populates the Northbound database
> + with a realistic ovn-kubernetes-style topology.
> +
> +The generated topology includes gateway routers, logical switches, NAT rules,
> +ACLs, DHCP/DHCPv6, DNS, load balancers, QoS rules, address sets, port groups,
> +static routes, and routing policies -- all with both IPv4 and IPv6 coverage.
> +After creation, incremental add/modify/delete operations exercise the northd
> +engine and flag unexpected recomputes.
> +
> +The benchmark must be run from inside the OVN sandbox. Run
> +``./ovn-benchmark.sh --help`` for the full list of options::
> +
> + $ make sandbox
> + $ ./ovn-benchmark.sh
> +
> +By default, the topology is generated through many individual OVSDB
> +transactions via ``ovn-benchmark.py``.
> +
> +.. note::
> +
> + The ``-f`` option loads a Northbound database file via ``ovsdb-client
> + restore`` instead, which applies the entire database as a single
> + transaction. This may yield different memory and timing results because
> + ``ovn-northd`` and ``ovn-controller`` process one large batch of changes
> + rather than reacting to each transaction incrementally.
> +
> OVN Upgrade Testing
> ~~~~~~~~~~~~~~~~~~~
>
> diff --git a/tutorial/automake.mk b/tutorial/automake.mk
> index 631208639..fdea2449c 100644
> --- a/tutorial/automake.mk
> +++ b/tutorial/automake.mk
> @@ -2,7 +2,9 @@ EXTRA_DIST += \
> tutorial/ovn-sandbox \
> tutorial/ovn-setup.sh \
> tutorial/ovn-lb-benchmark.sh \
> - tutorial/ovn-lb-benchmark.py
> + tutorial/ovn-lb-benchmark.py \
> + tutorial/ovn-benchmark.sh \
> + tutorial/ovn-benchmark.py
> sandbox: all
> cd $(srcdir)/tutorial && MAKE=$(MAKE) HAVE_OPENSSL=$(HAVE_OPENSSL) \
> ./ovn-sandbox -b $(abs_builddir) --ovs-src $(ovs_srcdir)
> --ovs-build $(ovs_builddir) $(SANDBOXFLAGS)
> diff --git a/tutorial/ovn-benchmark.py b/tutorial/ovn-benchmark.py
> new file mode 100755
> index 000000000..d2d4bb9fd
> --- /dev/null
> +++ b/tutorial/ovn-benchmark.py
> @@ -0,0 +1,1449 @@
> +#!/usr/bin/env python3
> +"""OVN memory and performance regression testing tool.
> +
> +Creates a broad OVN topology, then exercises incremental engine paths
> +by adding, modifying, and deleting resources. Designed to be run via
> +ovn-benchmark.sh, which measures peak memory, per-phase timing, and
> +checks for unexpected engine recomputes. (Run ./ovn-benchmark.sh
> +--help to see usage).
> +
> +Topology (simulates the default ovn-kubernetes topology):
> +
> + lsp-0-* lsp-1-* (Workload ports)
> + | |
> + ls-0 ls-1 (Logical Switches)
> + | |
> + s2c-0/c2s-0 s2c-1/c2s-1
> + \\ /
> + +--- cluster (LR) ---+ (Cluster Router)
> + |
> + sjc/rcj
> + |
> + +---- join (LS) ----+ (Join Switch)
> + / \\
> + j2gw-0/gw2j-0 j2gw-1/gw2j-1
> + | |
> + gw-0 gw-1 (Gateway Routers)
> +
> + Chassis binding (with batch size B, default n/10):
> + chassis-1: c2s-[0..B), gw-[0..B) (matches sandbox system-id)
> + chassis-2: c2s-[B..2B), gw-[B..2B)
> + ...
> +
> +Each node creates a dual-stack gateway router + logical switch pair
> +with:
> + - Dual-stack addressing (IPv4 10.x.x.x/24, IPv6 fd00:x::/64)
> + - NAT (SNAT/DNAT/DNAT_AND_SNAT), static routes, routing policies
> + - Configurable ports per switch with port security
> + - Security: Address sets, port groups, ACLs
> + - Services: DHCP, DHCPv6, DNS, load balancers
> + - QoS: Bandwidth limiting, DSCP marking
> +
> +Note: Uses explicit (non-templated) load balancers to maximize memory
> +usage for regression testing. For templated LB testing, see
> +ovn-lb-benchmark.py.
> +"""
> +
> +import argparse
> +import sys
> +
> +import ovs.db.idl
> +import ovs.poller
> +import ovs.vlog
> +from ovs.db import error
> +
> +vlog = ovs.vlog.Vlog('ovn-benchmark')
> +vlog.set_levels_from_string('console:warn')
> +vlog.init(None)
> +
> +SCHEMA = '../ovn-nb.ovsschema'
> +
> +
> +def die(msg):
> + sys.stderr.write(f'\nError: {msg}\n')
> + sys.exit(1)
> +
> +
> +def ip_node(i):
> + """Convert node index to two IP octets, supporting up to 65535 nodes."""
> + return f'{i >> 8}.{i & 0xff}'
> +
> +
> +def ip6_node(i):
> + """Convert node index to an IPv6 hextet for ULA addresses."""
> + return f'{i:x}'
> +
> +
> +def create_address_sets(idl, n):
> + """Create address sets for security groups."""
> + vlog.info(f'Creating address sets: web_servers ({n} v4 + {n} v6), '
> + f'db_servers ({n} v4 + {n} v6), app_servers ({n} v4 + {n} v6),
> '
> + f'trusted_networks_v4, trusted_networks_v6')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + web_as = txn.insert(idl.tables['Address_Set'])
> + web_as.name = 'web_servers'
> + web_as.addresses = (
> + [f'10.{ip_node(i)}.10' for i in range(n)]
> + + [f'fd00:{ip6_node(i)}::a' for i in range(n)])
> +
> + db_as = txn.insert(idl.tables['Address_Set'])
> + db_as.name = 'db_servers'
> + db_as.addresses = (
> + [f'10.{ip_node(i)}.20' for i in range(n)]
> + + [f'fd00:{ip6_node(i)}::14' for i in range(n)])
> +
> + app_as = txn.insert(idl.tables['Address_Set'])
> + app_as.name = 'app_servers'
> + app_as.addresses = (
> + [f'10.{ip_node(i)}.30' for i in range(n)]
> + + [f'fd00:{ip6_node(i)}::1e' for i in range(n)])
> +
> + trusted_as_v4 = txn.insert(idl.tables['Address_Set'])
> + trusted_as_v4.name = 'trusted_networks_v4'
> + trusted_as_v4.addresses = ['192.168.0.0/16', '172.16.0.0/12']
> +
> + trusted_as_v6 = txn.insert(idl.tables['Address_Set'])
> + trusted_as_v6.name = 'trusted_networks_v6'
> + trusted_as_v6.addresses = ['fc00::/7']
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create address sets ({txn.get_error()})')
> +
> +
> +def create_port_groups(idl):
> + """Create port groups for security group implementation."""
> + vlog.info('Creating port groups: web_tier, db_tier, app_tier')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + web_pg = txn.insert(idl.tables['Port_Group'])
> + web_pg.name = 'web_tier'
> +
> + db_pg = txn.insert(idl.tables['Port_Group'])
> + db_pg.name = 'db_tier'
> +
> + app_pg = txn.insert(idl.tables['Port_Group'])
> + app_pg.name = 'app_tier'
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create port groups ({txn.get_error()})')
> +
> +
> +def create_dhcp_options(idl, n):
> + """Create DHCP options for each subnet."""
> + for i in range(n):
> + vlog.info(f'Creating DHCP options for node {i}: '
> + f'10.{ip_node(i)}.0/24, fd00:{ip6_node(i)}::/64')
> + txn = ovs.db.idl.Transaction(idl)
> + dhcp_opts = txn.insert(idl.tables['DHCP_Options'])
> + dhcp_opts.cidr = f'10.{ip_node(i)}.0/24'
> + dhcp_opts.setkey('options', 'server_id', f'10.{ip_node(i)}.1')
> + dhcp_opts.setkey('options', 'server_mac', '00:00:00:00:00:01')
> + dhcp_opts.setkey('options', 'lease_time', '3600')
> + dhcp_opts.setkey('options', 'router', f'10.{ip_node(i)}.1')
> + dhcp_opts.setkey('options', 'dns_server', f'10.{ip_node(i)}.2')
> + dhcp_opts.setkey('options', 'domain_name', '"example.com"')
> + dhcp_opts.setkey('options', 'mtu', '1500')
> + dhcp_opts.setkey('external_ids', 'subnet', f'ls-{i}')
> +
> + dhcpv6_opts = txn.insert(idl.tables['DHCP_Options'])
> + dhcpv6_opts.cidr = f'fd00:{ip6_node(i)}::/64'
> + dhcpv6_opts.setkey('options', 'server_id',
> + '00:00:00:00:00:01')
> + dhcpv6_opts.setkey('options', 'dns_server',
> + f'fd00:{ip6_node(i)}::2')
> + dhcpv6_opts.setkey('options', 'domain_search',
> + '"example.com"')
> + dhcpv6_opts.setkey('external_ids', 'subnet',
> + f'ls-{i}-v6')
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create DHCP options for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def create_dhcp_assignments(idl, n):
> + """Assign DHCP options to logical switch ports."""
> + dhcp4 = {}
> + dhcp6 = {}
> + for row in idl.tables['DHCP_Options'].rows.values():
> + subnet = row.external_ids.get('subnet', '')
> + if subnet.endswith('-v6'):
> + dhcp6[subnet[:-3]] = row
> + else:
> + dhcp4[subnet] = row
> +
> + for i in range(n):
> + vlog.info(f'Assigning DHCP options for node {i}: '
> + f'ls-{i} ports -> 10.{ip_node(i)}.0/24 + '
> + f'fd00:{ip6_node(i)}::/64')
> + txn = ovs.db.idl.Transaction(idl)
> + opts4 = dhcp4.get(f'ls-{i}')
> + opts6 = dhcp6.get(f'ls-{i}')
> + for row in idl.tables['Logical_Switch_Port'].rows.values():
> + if row.name.startswith(f'lsp-{i}-'):
> + if opts4:
> + row.dhcpv4_options = opts4.uuid
> + if opts6:
> + row.dhcpv6_options = opts6.uuid
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to assign DHCP options for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def create_qos_rules(idl, n, switches):
> + """Create QoS rules for bandwidth limiting and DSCP marking."""
> + for i in range(n):
> + vlog.info(f'Creating QoS rules for node {i}: '
> + f'rate-limit on lsp-{i}-0, DSCP/mark on ls-{i}')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + ls = switches.get(f'ls-{i}')
> + if not ls:
> + die(f'Missing switch ls-{i}')
> +
> + qos_bw = txn.insert(idl.tables['QoS'])
> + qos_bw.priority = 100
> + qos_bw.direction = 'to-lport'
> + qos_bw.match = f'inport == "lsp-{i}-0"'
> + qos_bw.setkey('bandwidth', 'rate', 1000)
> + qos_bw.setkey('bandwidth', 'burst', 100)
> + qos_bw.setkey('external_ids', 'type', 'rate-limit')
> + ls.addvalue('qos_rules', qos_bw.uuid)
> +
> + qos_dscp = txn.insert(idl.tables['QoS'])
> + qos_dscp.priority = 200
> + qos_dscp.direction = 'from-lport'
> + qos_dscp.match = 'ip4 && tcp.dst == 22'
> + qos_dscp.setkey('action', 'dscp', 46)
> + qos_dscp.setkey('external_ids', 'type', 'dscp-marking')
> + ls.addvalue('qos_rules', qos_dscp.uuid)
> +
> + qos_mark = txn.insert(idl.tables['QoS'])
> + qos_mark.priority = 150
> + qos_mark.direction = 'from-lport'
> + qos_mark.match = 'ip4 && udp'
> + qos_mark.setkey('action', 'mark', 1)
> + qos_mark.setkey('external_ids', 'type', 'packet-marking')
> + ls.addvalue('qos_rules', qos_mark.uuid)
> +
> + qos_dscp_v6 = txn.insert(idl.tables['QoS'])
> + qos_dscp_v6.priority = 200
> + qos_dscp_v6.direction = 'from-lport'
> + qos_dscp_v6.match = 'ip6 && tcp.dst == 22'
> + qos_dscp_v6.setkey('action', 'dscp', 46)
> + qos_dscp_v6.setkey('external_ids', 'type',
> + 'dscp-marking-v6')
> + ls.addvalue('qos_rules', qos_dscp_v6.uuid)
> +
> + qos_mark_v6 = txn.insert(idl.tables['QoS'])
> + qos_mark_v6.priority = 150
> + qos_mark_v6.direction = 'from-lport'
> + qos_mark_v6.match = 'ip6 && udp'
> + qos_mark_v6.setkey('action', 'mark', 1)
> + qos_mark_v6.setkey('external_ids', 'type',
> + 'packet-marking-v6')
> + ls.addvalue('qos_rules', qos_mark_v6.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create QoS rules for node {i}
> ({txn.get_error()})')
> +
> +
> +def create_acls_for_port_group(idl, port_groups, pg_name, allowed_ports):
> + """Create ACLs for a specific port group."""
> + txn = ovs.db.idl.Transaction(idl)
> +
> + pg = port_groups.get(pg_name)
> + if not pg:
> + die(f'Missing port group {pg_name}')
> +
> + acl_allow_est = txn.insert(idl.tables['ACL'])
> + acl_allow_est.priority = 1100
> + acl_allow_est.direction = 'to-lport'
> + acl_allow_est.match = 'ct.est && !ct.rel && !ct.new && !ct.inv'
> + acl_allow_est.action = 'allow-related'
> + pg.addvalue('acls', acl_allow_est.uuid)
> +
> + acl_allow_rel = txn.insert(idl.tables['ACL'])
> + acl_allow_rel.priority = 1100
> + acl_allow_rel.direction = 'to-lport'
> + acl_allow_rel.match = 'ct.rel && !ct.est && !ct.new && !ct.inv'
> + acl_allow_rel.action = 'allow-related'
> + pg.addvalue('acls', acl_allow_rel.uuid)
> +
> + for port in allowed_ports:
> + acl_new = txn.insert(idl.tables['ACL'])
> + acl_new.priority = 1050
> + acl_new.direction = 'to-lport'
> + acl_new.match = f'ct.new && tcp.dst == {port}'
> + acl_new.action = 'allow-related'
> + pg.addvalue('acls', acl_new.uuid)
> +
> + acl_drop = txn.insert(idl.tables['ACL'])
> + acl_drop.priority = 1000
> + acl_drop.direction = 'to-lport'
> + acl_drop.match = f'outport == @{pg_name}'
> + acl_drop.action = 'drop'
> + pg.addvalue('acls', acl_drop.uuid)
> +
> + acl_arp = txn.insert(idl.tables['ACL'])
> + acl_arp.priority = 1010
> + acl_arp.direction = 'to-lport'
> + acl_arp.match = 'arp || nd'
> + acl_arp.action = 'allow'
> + pg.addvalue('acls', acl_arp.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create ACLs for {pg_name} ({txn.get_error()})')
> +
> +
> +def create_acls(idl, port_groups):
> + """Create comprehensive ACLs for port groups.
> +
> + ACL Priority Allocation:
> + 2500-2999: Security enforcement (anti-spoofing, etc.)
> + 2000-2499: Management access (SSH, ICMP, DHCP)
> + 1000-1499: Port group ACLs (security groups)
> + 1100: Connection tracking (established, related)
> + 1050: New connection per-port rules
> + 1010: ARP/ND allow
> + 1000: Default drop
> +
> + Switch ACLs (higher priority) can override port-group ACLs, with
> + security enforcement (anti-spoofing) taking highest priority.
> + """
> + vlog.info('Creating ACLs for port groups')
> + create_acls_for_port_group(idl, port_groups, 'web_tier', [80, 443])
> + create_acls_for_port_group(idl, port_groups, 'app_tier', [8080, 9000])
> + create_acls_for_port_group(idl, port_groups, 'db_tier', [5432, 3306])
> +
> +
> +def _add_acl(txn, idl, ls, priority, direction, match, action,
> + description=None):
> + """Create an ACL and attach it to a logical switch."""
> + acl = txn.insert(idl.tables['ACL'])
> + acl.priority = priority
> + acl.direction = direction
> + acl.match = match
> + acl.action = action
> + if description:
> + acl.setkey('external_ids', 'description', description)
> + ls.addvalue('acls', acl.uuid)
> +
> +
> +def create_switch_acls(idl, switch_name, node_id, switches):
> + """Create ACLs on a logical switch.
> +
> + Creates switch-level ACLs for SSH, ICMP, anti-spoofing, and DHCP.
> + Anti-spoofing (priority 2500) prevents VMs from using IPs outside
> + their assigned subnet, blocking IP address spoofing attacks.
> + """
> + txn = ovs.db.idl.Transaction(idl)
> +
> + ls = switches.get(switch_name)
> + if not ls:
> + die(f'Missing switch {switch_name}')
> +
> + _add_acl(txn, idl, ls, 2000, 'from-lport',
> + 'tcp.dst == 22 && ip4.src == 10.0.0.0/8',
> + 'allow', 'Allow SSH from internal')
> + _add_acl(txn, idl, ls, 1500, 'from-lport',
> + 'icmp4 || icmp6', 'allow')
> + _add_acl(txn, idl, ls, 2500, 'from-lport',
> + f'ip4.src != 10.{ip_node(node_id)}.0/24',
> + 'drop', 'Anti-spoofing')
> + _add_acl(txn, idl, ls, 2000, 'from-lport',
> + 'udp.src == 68 && udp.dst == 67', 'allow')
> +
> + _add_acl(txn, idl, ls, 2000, 'from-lport',
> + 'tcp.dst == 22 && ip6.src == fd00::/16',
> + 'allow', 'Allow SSH from internal (IPv6)')
> + _add_acl(txn, idl, ls, 2500, 'from-lport',
> + f'ip6.src != fd00:{ip6_node(node_id)}::/64',
> + 'drop', 'Anti-spoofing (IPv6)')
> + _add_acl(txn, idl, ls, 2000, 'from-lport',
> + 'udp.src == 546 && udp.dst == 547', 'allow')
> +
> + _add_acl(txn, idl, ls, 1800, 'to-lport',
> + 'ip4.dst == $web_servers && tcp.dst == 80',
> + 'allow-related', 'Allow HTTP to web servers')
> + _add_acl(txn, idl, ls, 1800, 'to-lport',
> + 'ip4.dst == $db_servers && tcp.dst == 5432',
> + 'allow-related', 'Allow Postgres to DB servers')
> + _add_acl(txn, idl, ls, 1800, 'to-lport',
> + 'ip4.dst == $app_servers && tcp.dst == 8080',
> + 'allow-related', 'Allow HTTP to app servers')
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create ACLs on switch {switch_name} '
> + f'({txn.get_error()})')
> +
> +
> +def create_dns_records(idl, n, switches):
> + """Create DNS records in the NB database."""
> + for i in range(n):
> + vlog.info(f'Creating DNS records for node {i}: '
> + f'web-{i}.example.com -> 10.{ip_node(i)}.10, '
> + f'app-{i}.example.com -> 10.{ip_node(i)}.30, '
> + f'db-{i}.example.com -> 10.{ip_node(i)}.20')
> + txn = ovs.db.idl.Transaction(idl)
> + dns = txn.insert(idl.tables['DNS'])
> + dns.setkey('records', f'web-{i}.example.com',
> + f'10.{ip_node(i)}.10 '
> + f'fd00:{ip6_node(i)}::a')
> + dns.setkey('records', f'app-{i}.example.com',
> + f'10.{ip_node(i)}.30 '
> + f'fd00:{ip6_node(i)}::1e')
> + dns.setkey('records', f'db-{i}.example.com',
> + f'10.{ip_node(i)}.20 '
> + f'fd00:{ip6_node(i)}::14')
> + dns.setkey('external_ids', 'zone', f'zone-{i}')
> +
> + ls = switches.get(f'ls-{i}')
> + if not ls:
> + die(f'Missing switch ls-{i}')
> + ls.addvalue('dns_records', dns.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create DNS records for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def create_nat_rules(idl, n, routers):
> + """Create NAT rules on routers.
> +
> + Creates SNAT, DNAT, and DNAT_AND_SNAT rules to exercise all NAT code
> paths.
> + """
> + for i in range(n):
> + vlog.info(f'Creating NAT rules on router gw-{i}: '
> + f'snat 10.{ip_node(i)}.0/24 -> 192.{ip_node(i)}.1, '
> + f'dnat .10 -> .10, dnat_and_snat .20 -> .20')
> + vlog.info(f' IPv6: snat fd00:{ip6_node(i)}::/64 -> '
> + f'fd01:{ip6_node(i)}::1, '
> + f'dnat ::a -> ::a, dnat_and_snat ::14 -> ::14')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + die(f'Missing router gw-{i}')
> +
> + nat_snat = txn.insert(idl.tables['NAT'])
> + nat_snat.type = 'snat'
> + nat_snat.logical_ip = f'10.{ip_node(i)}.0/24'
> + nat_snat.external_ip = f'192.{ip_node(i)}.1'
> + lr.addvalue('nat', nat_snat.uuid)
> +
> + nat_dnat = txn.insert(idl.tables['NAT'])
> + nat_dnat.type = 'dnat'
> + nat_dnat.logical_ip = f'10.{ip_node(i)}.10'
> + nat_dnat.external_ip = f'192.{ip_node(i)}.10'
> + nat_dnat.setkey('external_ids', 'service', 'web')
> + lr.addvalue('nat', nat_dnat.uuid)
> +
> + nat_dnat_and_snat = txn.insert(idl.tables['NAT'])
> + nat_dnat_and_snat.type = 'dnat_and_snat'
> + nat_dnat_and_snat.logical_ip = f'10.{ip_node(i)}.20'
> + nat_dnat_and_snat.external_ip = f'192.{ip_node(i)}.20'
> + nat_dnat_and_snat.setkey('external_ids', 'service', 'db')
> + lr.addvalue('nat', nat_dnat_and_snat.uuid)
> +
> + nat_snat_v6 = txn.insert(idl.tables['NAT'])
> + nat_snat_v6.type = 'snat'
> + nat_snat_v6.logical_ip = f'fd00:{ip6_node(i)}::/64'
> + nat_snat_v6.external_ip = f'fd01:{ip6_node(i)}::1'
> + lr.addvalue('nat', nat_snat_v6.uuid)
> +
> + nat_dnat_v6 = txn.insert(idl.tables['NAT'])
> + nat_dnat_v6.type = 'dnat'
> + nat_dnat_v6.logical_ip = f'fd00:{ip6_node(i)}::a'
> + nat_dnat_v6.external_ip = f'fd01:{ip6_node(i)}::a'
> + nat_dnat_v6.setkey('external_ids',
> + 'service', 'web-v6')
> + lr.addvalue('nat', nat_dnat_v6.uuid)
> +
> + nat_ds_v6 = txn.insert(idl.tables['NAT'])
> + nat_ds_v6.type = 'dnat_and_snat'
> + nat_ds_v6.logical_ip = f'fd00:{ip6_node(i)}::14'
> + nat_ds_v6.external_ip = f'fd01:{ip6_node(i)}::14'
> + nat_ds_v6.setkey('external_ids',
> + 'service', 'db-v6')
> + lr.addvalue('nat', nat_ds_v6.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create NAT rules for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def create_static_routes(idl, n, routers):
> + """Create static routes on routers."""
> + for i in range(n):
> + vlog.info(f'Creating static routes on router gw-{i}: '
> + f'default 0.0.0.0/0 -> 100.64.0.1, '
> + f'172.{ip_node(i)}.0/24 -> 10.{ip_node(i)}.254, '
> + f'192.0.2.0/24 -> discard')
> + vlog.info(f' IPv6: ::/0 -> fd01::1, '
> + f'fd02:{ip6_node(i)}::/64 -> fd00:{ip6_node(i)}::fe, '
> + f'2001:db8::/32 -> discard')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + die(f'Missing router gw-{i}')
> +
> + route_default = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_default.ip_prefix = '0.0.0.0/0'
> + route_default.nexthop = '100.64.0.1'
> + route_default.setkey('external_ids', 'type', 'default')
> + lr.addvalue('static_routes', route_default.uuid)
> +
> + route_specific = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_specific.ip_prefix = f'172.{ip_node(i)}.0/24'
> + route_specific.nexthop = f'10.{ip_node(i)}.254'
> + route_specific.setkey('external_ids', 'type', 'specific')
> + lr.addvalue('static_routes', route_specific.uuid)
> +
> + route_discard = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_discard.ip_prefix = '192.0.2.0/24'
> + route_discard.nexthop = 'discard'
> + route_discard.setkey('external_ids', 'type', 'blackhole')
> + lr.addvalue('static_routes', route_discard.uuid)
> +
> + route_default_v6 = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_default_v6.ip_prefix = '::/0'
> + route_default_v6.nexthop = 'fd01::1'
> + route_default_v6.setkey('external_ids', 'type',
> + 'default-v6')
> + lr.addvalue('static_routes', route_default_v6.uuid)
> +
> + route_specific_v6 = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_specific_v6.ip_prefix = (
> + f'fd02:{ip6_node(i)}::/64')
> + route_specific_v6.nexthop = (
> + f'fd00:{ip6_node(i)}::fe')
> + route_specific_v6.setkey('external_ids', 'type',
> + 'specific-v6')
> + lr.addvalue('static_routes',
> + route_specific_v6.uuid)
> +
> + route_discard_v6 = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route_discard_v6.ip_prefix = '2001:db8::/32'
> + route_discard_v6.nexthop = 'discard'
> + route_discard_v6.setkey('external_ids', 'type',
> + 'blackhole-v6')
> + lr.addvalue('static_routes',
> + route_discard_v6.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create static routes for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def create_routing_policies(idl, n, routers):
> + """Create routing policies on routers."""
> + for i in range(n):
> + vlog.info(f'Creating routing policies on router gw-{i}: '
> + f'reroute 10.{ip_node(i)}.0/24 -> '
> + f'10.{ip_node((i + 1) % n)}.1, '
> + f'allow $trusted_networks_v4')
> + vlog.info(f' IPv6: reroute fd00:{ip6_node(i)}::/64 -> '
> + f'fd00:{ip6_node((i + 1) % n)}::1, '
> + f'allow $trusted_networks_v6')
> + txn = ovs.db.idl.Transaction(idl)
> +
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + die(f'Missing router gw-{i}')
> +
> + policy_reroute = txn.insert(idl.tables['Logical_Router_Policy'])
> + policy_reroute.priority = 100
> + policy_reroute.match = f'ip4.src == 10.{ip_node(i)}.0/24'
> + policy_reroute.action = 'reroute'
> + policy_reroute.nexthops = [f'10.{ip_node((i + 1) % n)}.1']
> + policy_reroute.setkey('external_ids', 'policy',
> + 'traffic-engineering')
> + lr.addvalue('policies', policy_reroute.uuid)
> +
> + policy_allow = txn.insert(idl.tables['Logical_Router_Policy'])
> + policy_allow.priority = 50
> + policy_allow.match = 'ip4.dst == $trusted_networks_v4'
> + policy_allow.action = 'allow'
> + lr.addvalue('policies', policy_allow.uuid)
> +
> + policy_reroute_v6 = txn.insert(
> + idl.tables['Logical_Router_Policy'])
> + policy_reroute_v6.priority = 100
> + policy_reroute_v6.match = (
> + f'ip6.src == fd00:{ip6_node(i)}::/64')
> + policy_reroute_v6.action = 'reroute'
> + policy_reroute_v6.nexthops = [
> + f'fd00:{ip6_node((i + 1) % n)}::1']
> + policy_reroute_v6.setkey(
> + 'external_ids', 'policy',
> + 'traffic-engineering-v6')
> + lr.addvalue('policies',
> + policy_reroute_v6.uuid)
> +
> + policy_allow_v6 = txn.insert(
> + idl.tables['Logical_Router_Policy'])
> + policy_allow_v6.priority = 50
> + policy_allow_v6.match = (
> + 'ip6.dst == $trusted_networks_v6')
> + policy_allow_v6.action = 'allow'
> + lr.addvalue('policies', policy_allow_v6.uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create routing policies for node {i} '
> + f'({txn.get_error()})')
> +
> +
> +def _add_ports(txn, idl, s, i, ports_per_switch):
> + for p in range(ports_per_switch):
> + lsp = txn.insert(idl.tables['Logical_Switch_Port'])
> + lsp.name = f'lsp-{i}-{p}'
> + # Offset by 10 to match the IP's +10 offset; wraps at 256 but
> + # the port cap (244) means this only matters at the boundary.
> + mac_byte = (p + 10) % 256
> + mac = (f'00:00:{i >> 8:02x}:{i & 0xff:02x}'
> + f':{p:02x}:{mac_byte:02x}')
> + ip = f'10.{ip_node(i)}.{10 + p}'
> + ip6 = f'fd00:{ip6_node(i)}::{10 + p:x}'
> + lsp.addresses = [f'{mac} {ip} {ip6}']
> + lsp.port_security = [f'{mac} {ip} {ip6}']
> + vlog.info(f' lsp-{i}-{p}: {mac} {ip} {ip6}')
> + lsp.setkey('external_ids', 'vm-id', f'vm-{i}-{p}')
> +
> + if p % 3 == 0:
> + lsp.setkey('external_ids', 'tier', 'web')
> + elif p % 3 == 1:
> + lsp.setkey('external_ids', 'tier', 'app')
> + else:
> + lsp.setkey('external_ids', 'tier', 'db')
> +
> + s.addvalue('ports', lsp.uuid)
> +
> +
> +def _add_node(txn, idl, i, chassis, cluster_rtr, join_sw, lbg,
> + ports_per_switch):
> + gwr = txn.insert(idl.tables['Logical_Router'])
> + gwr.name = f'gw-{i}'
> + gwr.addvalue('load_balancer_group', lbg.uuid)
> + gwr.setkey('options', 'chassis', chassis)
> +
> + gwr2join = txn.insert(idl.tables['Logical_Router_Port'])
> + gwr2join.name = f'gw2j-{i}'
> + gwr2join.mac = f'00:00:01:{i >> 8:02x}:{i & 0xff:02x}:01'
> + gwr2join.networks = [f'100.64.{ip_node(i)}/16',
> + f'fd01:{ip6_node(i)}::1/48']
> + gwr.addvalue('ports', gwr2join.uuid)
> +
> + join2gwr = txn.insert(idl.tables['Logical_Switch_Port'])
> + join2gwr.name = f'j2gw-{i}'
> + join2gwr.type = 'router'
> + join2gwr.addresses = ['router']
> + join2gwr.setkey('options', 'router-port', gwr2join.name)
> + join_sw.addvalue('ports', join2gwr.uuid)
> +
> + s = txn.insert(idl.tables['Logical_Switch'])
> + s.name = f'ls-{i}'
> + s.addvalue('load_balancer_group', lbg.uuid)
> + s.setkey('other_config', 'subnet', f'10.{ip_node(i)}.0/24')
> + s.setkey('other_config', 'mcast_snoop', 'true')
> +
> + cluster2s = txn.insert(idl.tables['Logical_Router_Port'])
> + cluster2s.name = f'c2s-{i}'
> + cluster2s.mac = '00:00:00:00:00:01'
> + cluster2s.networks = [f'10.{ip_node(i)}.1/24',
> + f'fd00:{ip6_node(i)}::1/64']
> + cluster_rtr.addvalue('ports', cluster2s.uuid)
> +
> + gw_chassis = txn.insert(idl.tables['Gateway_Chassis'])
> + gw_chassis.name = f'{cluster2s.name}-{chassis}'
> + gw_chassis.chassis_name = chassis
> + gw_chassis.priority = 1
> + cluster2s.addvalue('gateway_chassis', gw_chassis.uuid)
> +
> + s2cluster = txn.insert(idl.tables['Logical_Switch_Port'])
> + s2cluster.name = f's2c-{i}'
> + s2cluster.type = 'router'
> + s2cluster.addresses = ['router']
> + s2cluster.setkey('options', 'router-port', cluster2s.name)
> + s.addvalue('ports', s2cluster.uuid)
> +
> + _add_ports(txn, idl, s, i, ports_per_switch)
> +
> +
> +def create_topology(idl, n, ports_per_switch, batch_size):
> + """Create the basic topology with routers, switches, and ports."""
> + vlog.info('Creating topology')
> + txn = ovs.db.idl.Transaction(idl)
> + lbg = txn.insert(idl.tables['Load_Balancer_Group'])
> + lbg.name = 'lbg'
We never add load balancers through this LB group. That's something
that would be nice to test for too. But I don't want to block this
patch set any longer. Would you mind sending a follow up for that?
> +
> + vlog.info('Creating join switch')
> + join_sw = txn.insert(idl.tables['Logical_Switch'])
> + join_sw.name = 'join'
> +
> + cluster_rtr = txn.insert(idl.tables['Logical_Router'])
> + cluster_rtr.name = 'cluster'
> +
> + rcj = txn.insert(idl.tables['Logical_Router_Port'])
> + rcj.name = 'rcj'
> + rcj.mac = '00:00:00:00:00:01'
> + rcj.networks = ['100.64.0.1/16', 'fd01::1/48']
> + cluster_rtr.addvalue('ports', rcj.uuid)
> +
> + sjc = txn.insert(idl.tables['Logical_Switch_Port'])
> + sjc.name = 'sjc'
> + sjc.type = 'router'
> + sjc.addresses = ['router']
> + sjc.setkey('options', 'router-port', 'rcj')
> + join_sw.addvalue('ports', sjc.uuid)
> +
> + for i in range(n):
> + chassis = f'chassis-{i // batch_size + 1}'
> + vlog.info(f'Provisioning node {i}: gw-{i} + ls-{i}, '
> + f'10.{ip_node(i)}.0/24, fd00:{ip6_node(i)}::/64, '
> + f'{chassis}')
> + _add_node(txn, idl, i, chassis, cluster_rtr, join_sw, lbg,
> + ports_per_switch)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create topology ({txn.get_error()})')
> +
> +
> +def create_port_group_assignments(idl, port_groups):
> + """Assign logical switch ports to port groups based on tier."""
> + tier_ports = {'web': [], 'app': [], 'db': []}
> +
> + for row in idl.tables['Logical_Switch_Port'].rows.values():
> + tier = row.external_ids.get('tier')
> + if tier in tier_ports:
> + tier_ports[tier].append(row.uuid)
> +
> + vlog.info(f'Assigning ports to port groups: '
> + f'{len(tier_ports["web"])} web, '
> + f'{len(tier_ports["app"])} app, '
> + f'{len(tier_ports["db"])} db')
> +
> + txn = ovs.db.idl.Transaction(idl)
> + tier_to_group = {'web': 'web_tier', 'app': 'app_tier', 'db': 'db_tier'}
> + for tier, group_name in tier_to_group.items():
> + pg = port_groups.get(group_name)
> + if not pg:
> + die(f'Missing port group {group_name}')
> + for port_uuid in tier_ports[tier]:
> + pg.addvalue('ports', port_uuid)
> +
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to assign ports to groups ({txn.get_error()})')
> +
> +
> +def create_explicit_lbs(idl, n, n_vips, n_backends, routers, switches):
> + """Create explicit (non-templated) load balancers.
> +
> + Uses explicit (non-templated) LBs to maximize memory usage for
> + regression testing.
> + """
> + for i in range(n):
> + lr = routers.get(f'gw-{i}')
> + ls = switches.get(f'ls-{i}')
> + if not lr or not ls:
> + die(f'Missing router or switch for node {i}')
> + for j in range(n_vips):
> + txn = ovs.db.idl.Transaction(idl)
> + port = j + 1
> + vlog.info(f'Creating LB lb-{j}-{i}: '
> + f'42.42.{ip_node(i)}:{port} -> '
> + f'{n_backends} backends')
> + # Spread VIP index across two octets to stay within 0-249.
> + # Unlike ip_node() which uses >>8/&0xff for node addresses,
> + # this uses //250/%250 because VIP counts are small and 250
> + # avoids addresses that look like broadcast/network octets.
> + # Backend addresses depend only on j (not i), so every node
> + # shares the same backend set -- intentional, since we are
> + # testing memory, not realistic traffic distribution.
> + j1 = (j + 1) // 250
> + j2 = (j + 1) % 250
> +
> + lb = txn.insert(idl.tables['Load_Balancer'])
> + lb.name = f'lb-{j}-{i}'
> + lb.setkey('vips', f'42.42.{ip_node(i)}:{port}',
> + ','.join(f'42.{k}.{j1}.{j2}:{port}'
> + for k in range(n_backends)))
> + lb.setkey(
> + 'vips',
> + f'[fd42::{ip6_node(i)}:{j:x}]:{port}',
> + ','.join(f'[fd42:{k:x}::{j1:x}:{j2:x}]:{port}'
> + for k in range(n_backends)))
> + lb.protocol = 'tcp'
> + lr.addvalue('load_balancer', lb.uuid)
> + ls.addvalue('load_balancer', lb.uuid)
> + if txn.commit_block() != ovs.db.idl.Transaction.SUCCESS:
> + die(f'Failed to create LB ({txn.get_error()})')
> +
> +
> +def connect_idl(remote):
> + """Connect to the NB database and return the IDL."""
> + schema_helper = ovs.db.idl.SchemaHelper(SCHEMA)
> + schema_helper.register_all()
> + idl = ovs.db.idl.Idl(remote, schema_helper, leader_only=False)
> +
> + while idl.change_seqno == 0 and not idl.run():
> + poller = ovs.poller.Poller()
> + idl.wait(poller)
> + poller.block()
> + return idl
> +
> +
> +def commit(txn, description):
> + """Commit a transaction, accepting both SUCCESS and UNCHANGED."""
> + status = txn.commit_block()
> + if status not in (ovs.db.idl.Transaction.SUCCESS,
> + ovs.db.idl.Transaction.UNCHANGED):
> + die(f'{description} ({txn.get_error()})')
> +
> +
> +def add_lsps(idl, n, ports_per_switch):
> + """Add one new port to ~10% of switches."""
> + if ports_per_switch < 1:
> + vlog.info('Skipping add_lsps: ports_per_switch is 0')
> + return
> + count = max(1, n // 10)
> + switches = {row.name: row
> + for row in idl.tables['Logical_Switch'].rows.values()}
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + p = ports_per_switch
> + for i in range(count):
> + ls = switches.get(f'ls-{i}')
> + if not ls:
> + continue
> + lsp = txn.insert(idl.tables['Logical_Switch_Port'])
> + lsp.name = f'lsp-{i}-{p}'
> + mac_byte = (p + 10) % 256
> + mac = (f'00:00:{i >> 8:02x}:{i & 0xff:02x}'
> + f':{p:02x}:{mac_byte:02x}')
> + ip = f'10.{ip_node(i)}.{10 + p}'
> + ip6 = f'fd00:{ip6_node(i)}::{10 + p:x}'
> + lsp.addresses = [f'{mac} {ip} {ip6}']
> + lsp.port_security = [f'{mac} {ip} {ip6}']
> + lsp.setkey('external_ids', 'vm-id', f'vm-{i}-{p}')
> + lsp.setkey('external_ids', 'tier', 'web')
> + ls.addvalue('ports', lsp.uuid)
> + added += 1
> + commit(txn, f'Failed to add {count} LSPs')
> + vlog.info(f'Added {added} LSPs '
> + f'(lsp-0-{p}..lsp-{count-1}-{p}, '
> + f'10.x.x.{10 + p})')
> +
> +
> +def add_nat(idl, n):
> + """Add one DNAT rule to ~10% of routers."""
> + count = max(1, n // 10)
> + routers = {row.name: row
> + for row in idl.tables['Logical_Router'].rows.values()}
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + for i in range(count):
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + continue
> + nat = txn.insert(idl.tables['NAT'])
> + nat.type = 'dnat'
> + nat.logical_ip = f'10.{ip_node(i)}.30'
> + nat.external_ip = f'192.{ip_node(i)}.30'
> + nat.setkey('external_ids', 'service', 'bench')
> + lr.addvalue('nat', nat.uuid)
> + added += 1
> + commit(txn, f'Failed to add {count} NAT rules (service=bench)')
> + vlog.info(f'Added {added} DNAT rules '
> + f'(service=bench, .30 -> .30)')
> +
> +
> +def add_routes(idl, n):
> + """Add one specific route to ~10% of routers."""
> + count = max(1, n // 10)
> + routers = {row.name: row
> + for row in idl.tables['Logical_Router'].rows.values()}
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + for i in range(count):
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + continue
> + route = txn.insert(
> + idl.tables['Logical_Router_Static_Route'])
> + route.ip_prefix = f'10.{ip_node(i)}.128/25'
> + route.nexthop = f'10.{ip_node(i)}.254'
> + route.setkey('external_ids', 'type', 'bench')
> + lr.addvalue('static_routes', route.uuid)
> + added += 1
> + commit(txn, f'Failed to add {count} static routes (type=bench)')
> + vlog.info(f'Added {added} static routes '
> + f'(10.x.x.128/25 -> 10.x.x.254)')
> +
> +
> +def add_policies(idl, n):
> + """Add one allow policy to ~10% of routers."""
> + count = max(1, n // 10)
> + routers = {row.name: row
> + for row in idl.tables['Logical_Router'].rows.values()}
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + for i in range(count):
> + lr = routers.get(f'gw-{i}')
> + if not lr:
> + continue
> + policy = txn.insert(idl.tables['Logical_Router_Policy'])
> + policy.priority = 75
> + policy.match = f'ip4.src == 10.{ip_node(i)}.128/25'
> + policy.action = 'allow'
> + policy.setkey('external_ids', 'source', 'bench')
> + lr.addvalue('policies', policy.uuid)
> + added += 1
> + commit(txn, f'Failed to add {count} routing policies (source=bench)')
> + vlog.info(f'Added {added} routing policies '
> + f'(priority=75, allow 10.x.x.128/25)')
> +
> +
> +def modify_nat(idl, n):
> + """Change external_ip on IPv4 SNAT rules for ~10% of routers."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['NAT'].rows.values():
> + if modified >= count:
> + break
> + if row.type == 'snat' and row.logical_ip.startswith('10.'):
> + prefix, last_octet = row.external_ip.rsplit('.', 1)
> + row.external_ip = f'{prefix}.{int(last_octet) + 1}'
> + modified += 1
> + commit(txn, f'Failed to modify {count} SNAT external_ips (.1 -> .2)')
> + vlog.info(f'Modified {modified} SNAT external_ips (.1 -> .2)')
> +
> +
> +def modify_routes(idl, n):
> + """Change nexthop on IPv4 default routes for ~10% of routers."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables[
> + 'Logical_Router_Static_Route'].rows.values():
> + if modified >= count:
> + break
> + if (row.external_ids.get('type') == 'default'
> + and row.ip_prefix == '0.0.0.0/0'):
> + row.nexthop = '100.64.0.2'
> + modified += 1
> + commit(txn,
> + f'Failed to modify {count} default routes '
> + f'(nexthop 100.64.0.1 -> .2)')
> + vlog.info(f'Modified {modified} default route nexthops '
> + f'(100.64.0.1 -> 100.64.0.2)')
> +
> +
> +def modify_lbs(idl, n):
> + """Change protocol on first LB for ~10% of nodes."""
> + count = max(1, n // 10)
> + targets = set(f'lb-0-{i}' for i in range(count))
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['Load_Balancer'].rows.values():
> + if row.name in targets:
> + row.protocol = 'udp' if row.protocol == 'tcp' else 'tcp'
> + modified += 1
> + commit(txn,
> + f'Failed to modify {count} LB protocols (tcp <-> udp)')
> + vlog.info(f'Modified {modified} LB protocols '
> + f'(lb-0-0..lb-0-{count-1}, tcp <-> udp)')
> +
> +
> +def modify_addr_sets(idl, n):
> + """Add ~10% of n addresses to the web_servers address set."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + for row in idl.tables['Address_Set'].rows.values():
> + if row.name == 'web_servers':
> + for i in range(count):
> + row.addvalue('addresses', f'10.{ip_node(i)}.200')
> + added = count
> + break
> + commit(txn, f'Failed to modify web_servers address set '
> + f'(+= {count} addresses)')
> + vlog.info(f'Added {added} addresses to web_servers '
> + f'(10.0.0.200..10.{ip_node(count-1)}.200)')
> +
> +
> +def modify_port_groups(idl, n):
> + """Add ~10% of app-tier ports to the web_tier port group."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + added = 0
> + app_ports = []
> + for row in idl.tables['Logical_Switch_Port'].rows.values():
> + if row.external_ids.get('tier') == 'app':
> + app_ports.append(row)
> + if len(app_ports) >= count:
> + break
> + web_tier = None
> + for row in idl.tables['Port_Group'].rows.values():
> + if row.name == 'web_tier':
> + web_tier = row
> + break
> + if web_tier:
> + for port in app_ports:
> + web_tier.addvalue('ports', port.uuid)
> + added += 1
> + commit(txn, f'Failed to modify web_tier port group '
> + f'(+= {count} ports)')
> + vlog.info(f'Added {added} app-tier ports to web_tier')
> +
> +
> +def modify_port_security(idl, n):
> + """Add an extra IP to port_security on ~10% of LSPs."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['Logical_Switch_Port'].rows.values():
> + if modified >= count:
> + break
> + if not row.name.startswith('lsp-') or not row.name.endswith('-1'):
> + continue
> + parts = row.name.split('-')
> + i = int(parts[1])
> + if i >= count:
> + continue
> + ps = list(row.port_security)
> + if ps:
> + ps[0] = ps[0] + f' 10.{ip_node(i)}.200'
> + row.port_security = ps
> + modified += 1
> + commit(txn,
> + f'Failed to modify {count} LSP port_security '
> + f'(+= 10.x.x.200)')
> + vlog.info(f'Modified {modified} LSP port_security '
> + f'(lsp-0-1..lsp-{count-1}-1, += 10.x.x.200)')
> +
> +
> +def modify_dhcp(idl, n):
> + """Change lease_time on ~10% of DHCPv4 option sets."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['DHCP_Options'].rows.values():
> + if modified >= count:
> + break
> + subnet = row.external_ids.get('subnet', '')
> + if subnet.startswith('ls-') and not subnet.endswith('-v6'):
> + parts = subnet.split('-')
> + i = int(parts[1])
> + if i < count:
> + row.setkey('options', 'lease_time', '7200')
> + modified += 1
> + commit(txn,
> + f'Failed to modify {count} DHCP lease_time '
> + f'(3600 -> 7200)')
> + vlog.info(f'Modified {modified} DHCP lease_time '
> + f'(3600 -> 7200)')
> +
> +
> +def modify_dns(idl, n):
> + """Add a DNS record to ~10% of nodes."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['DNS'].rows.values():
> + if modified >= count:
> + break
> + zone = row.external_ids.get('zone', '')
> + if zone.startswith('zone-'):
> + i = int(zone.split('-')[1])
> + if i < count:
> + row.setkey('records', f'bench-{i}.example.com',
> + f'10.{ip_node(i)}.200')
> + modified += 1
> + commit(txn,
> + f'Failed to modify {count} DNS records '
> + f'(+= bench-x.example.com)')
> + vlog.info(f'Modified {modified} DNS records '
> + f'(+= bench-x.example.com -> 10.x.x.200)')
> +
> +
> +def delete_acls(idl, n):
> + """Delete 'Allow SSH from internal' ACL from ~10% of switches."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(idl.tables['ACL'].rows.values()):
> + if deleted >= count:
> + break
> + desc = row.external_ids.get('description', '')
> + if desc == 'Allow SSH from internal':
> + row.delete()
> + deleted += 1
> + commit(txn,
> + f'Failed to delete {count} ACLs '
> + f'(Allow SSH from internal)')
> + vlog.info(f'Deleted {deleted} ACLs '
> + f'(Allow SSH from internal)')
> +
> +
> +def delete_policies(idl, n):
> + """Delete IPv4 allow policies from ~10% of routers."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(
> + idl.tables['Logical_Router_Policy'].rows.values()):
> + if deleted >= count:
> + break
> + if (row.priority == 50
> + and 'trusted_networks_v4' in row.match):
> + row.delete()
> + deleted += 1
> + commit(txn,
> + f'Failed to delete {count} routing policies '
> + f'(priority=50, trusted_networks_v4)')
> + vlog.info(f'Deleted {deleted} routing policies '
> + f'(priority=50, trusted_networks_v4)')
> +
> +
> +def delete_qos(idl, n):
> + """Delete rate-limit QoS rules from ~10% of switches."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(idl.tables['QoS'].rows.values()):
> + if deleted >= count:
> + break
> + if row.external_ids.get('type') == 'rate-limit':
> + row.delete()
> + deleted += 1
> + commit(txn,
> + f'Failed to delete {count} QoS rules (type=rate-limit)')
> + vlog.info(f'Deleted {deleted} QoS rules (type=rate-limit)')
> +
> +
> +def modify_lsps(idl, n):
> + """Modify addresses on ~10% of LSPs to exercise incremental
> processing."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['Logical_Switch_Port'].rows.values():
> + if modified >= count:
> + break
> + name = row.name
> + if not name.startswith('lsp-') or not name.endswith('-0'):
> + continue
> + parts = name.split('-')
> + i = int(parts[1])
> + if i >= count:
> + continue
> + mac = f'00:00:{i >> 8:02x}:{i & 0xff:02x}:00:0a'
> + ip = f'10.{ip_node(i)}.210'
> + ip6 = f'fd00:{ip6_node(i)}::d2'
> + row.addresses = [f'{mac} {ip} {ip6}']
> + row.port_security = [f'{mac} {ip} {ip6}']
> + modified += 1
> + commit(txn, f'Failed to modify {count} LSP addresses (.10 -> .210)')
> + vlog.info(f'Modified {modified} LSP addresses '
> + f'(lsp-0-0..lsp-{count-1}-0, .10 -> .210)')
> +
> +
> +def modify_acls(idl, n):
> + """Modify anti-spoofing ACLs on ~10% of switches."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + modified = 0
> + for row in idl.tables['ACL'].rows.values():
> + if modified >= count:
> + break
> + desc = row.external_ids.get('description', '')
> + if desc == 'Anti-spoofing' and '/24' in row.match:
> + row.match = row.match.replace('/24', '/23')
> + modified += 1
> + commit(txn, f'Failed to modify {count} anti-spoofing ACLs (/24 -> /23)')
> + vlog.info(f'Modified {modified} anti-spoofing ACLs (/24 -> /23)')
> +
> +
> +def delete_lsps(idl, n, ports_per_switch):
> + """Delete the last port from ~10% of switches."""
> + if ports_per_switch < 1:
> + vlog.info('Skipping delete_lsps: no ports to delete')
> + return
> + count = max(1, n // 10)
> + last_port = ports_per_switch - 1
> + targets = set(f'lsp-{i}-{last_port}' for i in range(count))
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(idl.tables['Logical_Switch_Port'].rows.values()):
> + if row.name in targets:
> + row.delete()
> + deleted += 1
> + commit(txn, f'Failed to delete {count} LSPs '
> + f'(lsp-0-{last_port}..lsp-{count-1}-{last_port})')
> + vlog.info(f'Deleted {deleted} LSPs '
> + f'(lsp-0-{last_port}..lsp-{count-1}-{last_port})')
> +
> +
> +def delete_nat(idl, n):
> + """Delete IPv4 DNAT rules from ~10% of routers."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(idl.tables['NAT'].rows.values()):
> + if deleted >= count:
> + break
> + if (row.type == 'dnat'
> + and row.external_ids.get('service') == 'web'):
> + row.delete()
> + deleted += 1
> + commit(txn, f'Failed to delete {count} IPv4 DNAT rules (service=web)')
> + vlog.info(f'Deleted {deleted} IPv4 DNAT rules (service=web)')
> +
> +
> +def delete_routes(idl, n):
> + """Delete IPv4 specific routes from ~10% of routers."""
> + count = max(1, n // 10)
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(
> + idl.tables['Logical_Router_Static_Route'].rows.values()):
> + if deleted >= count:
> + break
> + if row.external_ids.get('type') == 'specific':
> + row.delete()
> + deleted += 1
> + commit(txn, f'Failed to delete {count} IPv4 specific routes')
> + vlog.info(f'Deleted {deleted} IPv4 specific routes')
> +
> +
> +def delete_lbs(idl, n):
> + """Delete one load balancer from ~10% of nodes."""
> + count = max(1, n // 10)
> + targets = set(f'lb-0-{i}' for i in range(count))
> + txn = ovs.db.idl.Transaction(idl)
> + deleted = 0
> + for row in list(idl.tables['Load_Balancer'].rows.values()):
> + if row.name in targets:
> + row.delete()
> + deleted += 1
> + commit(txn, f'Failed to delete {count} load balancers '
> + f'(lb-0-0..lb-0-{count-1})')
> + vlog.info(f'Deleted {deleted} load balancers '
> + f'(lb-0-0..lb-0-{count-1})')
> +
> +
> +def run(remote, n, n_vips, n_backends, ports_per_switch, batch_size):
> + """Main execution function."""
> + idl = connect_idl(remote)
> +
> + # Check if database is clean before proceeding
> + if (len(idl.tables['Load_Balancer_Group'].rows) > 0 or
> + len(idl.tables['Logical_Switch'].rows) > 0 or
> + len(idl.tables['Logical_Router'].rows) > 0):
> + die('Database is not empty. Please restart the sandbox or clear the '
> + 'database before running this script.')
> +
> + create_topology(idl, n, ports_per_switch, batch_size)
> +
> + # Build lookup dictionaries for O(1) access to switches and routers
> + switches = {row.name: row
> + for row in idl.tables['Logical_Switch'].rows.values()}
> + routers = {row.name: row
> + for row in idl.tables['Logical_Router'].rows.values()}
> +
> + create_address_sets(idl, n)
> + create_port_groups(idl)
> + port_groups = {row.name: row
> + for row in idl.tables['Port_Group'].rows.values()}
> + create_port_group_assignments(idl, port_groups)
> + create_dhcp_options(idl, n)
> + create_dhcp_assignments(idl, n)
> + create_dns_records(idl, n, switches)
> + create_nat_rules(idl, n, routers)
> + create_static_routes(idl, n, routers)
> + create_acls(idl, port_groups)
> + for i in range(n):
> + create_switch_acls(idl, f'ls-{i}', i, switches)
> + create_qos_rules(idl, n, switches)
> + create_routing_policies(idl, n, routers)
> + create_explicit_lbs(idl, n, n_vips, n_backends, routers, switches)
> +
> +
> +PHASES = [
> + 'create',
> + 'add-lsps', 'add-nat', 'add-routes', 'add-policies',
> + 'modify-lsps', 'modify-acls', 'modify-nat', 'modify-routes',
> + 'modify-lbs', 'modify-addr-sets', 'modify-pg', 'modify-port-sec',
> + 'modify-dhcp', 'modify-dns',
> + 'delete-lsps', 'delete-nat', 'delete-routes', 'delete-lbs',
> + 'delete-acls', 'delete-policies', 'delete-qos',
> +]
> +
> +
> +def main():
> + """Parse arguments and run the benchmark."""
> + parser = argparse.ArgumentParser(
> + description='OVN benchmark: create topology, then exercise '
> + 'incremental engine with add/modify/delete operations'
> + )
> + parser.add_argument(
> + 'phase', nargs='?', default='create', choices=PHASES,
> + help='Benchmark phase to run (default: create)',
> + )
> + parser.add_argument(
> + '-r', '--remote', required=True, help='NB connection string'
> + )
> + parser.add_argument(
> + '-n', '--nodes', type=int, required=True, help='Number of nodes'
> + )
> + parser.add_argument(
> + '-p',
> + '--ports-per-switch',
> + type=int,
> + default=9,
> + help='Number of logical switch ports per switch (default: 9, '
> + 'provides 3 ports per tier)',
> + )
> + parser.add_argument(
> + '-v', '--vips', type=int, default=5,
> + help='Number of LB VIPs per node (default: 5)'
> + )
> + parser.add_argument(
> + '-B',
> + '--backends',
> + type=int,
> + default=5,
> + help='Number backends per VIP (default: 5)',
> + )
> + parser.add_argument(
> + '-b',
> + '--batch-size',
> + type=int,
> + default=0,
> + help='Nodes per chassis (default: n/10)',
> + )
> + parser.add_argument(
> + '-d', '--debug',
> + action='store_true',
> + help='Enable debug output (show info messages)',
> + )
> + args = parser.parse_args()
> +
> + if args.batch_size <= 0:
> + args.batch_size = max(1, args.nodes // 10)
> +
> + if args.nodes < 1:
> + die('node count must be at least 1')
> + # Node index is split across two IP octets via ip_node(), so the
> + # maximum is 2^16 - 1.
> + if args.nodes > 65535:
> + die('maximum supported node count is 65535')
> +
> + if args.ports_per_switch < 0:
> + die('ports per switch must not be negative')
> + # Port IPs start at 10.x.y.10 (see _add_ports), so the highest
> + # address is 10.x.y.(10 + ports_per_switch - 1). Cap at 245 to
> + # keep that <= 254 (valid IPv4 octet).
> + if args.ports_per_switch > 245:
> + die('maximum supported ports per switch is 245')
> +
> + if args.vips < 0:
> + die('VIP count must not be negative')
> + # VIP index is split via (j+1)//250 and (j+1)%250; the quotient
> + # must fit in one octet, so max VIPs = 250 * 256 - 1 = 63999.
> + if args.vips > 63999:
> + die('maximum supported VIPs is 63999')
> +
> + if args.backends < 0:
> + die('backend count must not be negative')
> + # Backend addresses use 42.{k}.j1.j2; k must fit in one octet.
> + if args.backends > 255:
> + die('maximum supported backends is 255')
> +
> + if args.debug:
> + vlog.set_levels_from_string('console:info')
> +
> + if args.phase == 'create':
> + n_chassis = ((args.nodes + args.batch_size - 1)
> + // args.batch_size)
> + print(f'\n=== OVN Benchmark Configuration ===')
> + print(f'Nodes: {args.nodes}'
> + f' ({args.nodes} routers + {args.nodes} switches)')
> + print(f'Ports per switch: {args.ports_per_switch} '
> + f'({args.ports_per_switch * args.nodes} total ports)')
> + print(f'Load balancer VIPs per node: {args.vips} '
> + f'({args.vips * args.nodes} total VIPs)')
> + print(f'Backends per VIP: {args.backends}')
> + print(f'Total load balancers: '
> + f'{args.nodes * args.vips}')
> + print(f'Nodes per chassis (batch): '
> + f'{args.batch_size} '
> + f'({n_chassis} chassis)')
> + print(f'Debug logging: '
> + f'{"enabled" if args.debug else "disabled"}')
> + print('===================================\n')
> +
> + run(args.remote, args.nodes, args.vips, args.backends,
> + args.ports_per_switch, args.batch_size)
> + else:
> + idl = connect_idl(args.remote)
> + n = args.nodes
> + if args.phase == 'add-lsps':
> + add_lsps(idl, n, args.ports_per_switch)
> + elif args.phase == 'add-nat':
> + add_nat(idl, n)
> + elif args.phase == 'add-routes':
> + add_routes(idl, n)
> + elif args.phase == 'add-policies':
> + add_policies(idl, n)
> + elif args.phase == 'modify-lsps':
> + modify_lsps(idl, n)
> + elif args.phase == 'modify-acls':
> + modify_acls(idl, n)
> + elif args.phase == 'modify-nat':
> + modify_nat(idl, n)
> + elif args.phase == 'modify-routes':
> + modify_routes(idl, n)
> + elif args.phase == 'modify-lbs':
> + modify_lbs(idl, n)
> + elif args.phase == 'modify-addr-sets':
> + modify_addr_sets(idl, n)
> + elif args.phase == 'modify-pg':
> + modify_port_groups(idl, n)
> + elif args.phase == 'modify-port-sec':
> + modify_port_security(idl, n)
> + elif args.phase == 'modify-dhcp':
> + modify_dhcp(idl, n)
> + elif args.phase == 'modify-dns':
> + modify_dns(idl, n)
> + elif args.phase == 'delete-lsps':
> + delete_lsps(idl, n, args.ports_per_switch)
> + elif args.phase == 'delete-nat':
> + delete_nat(idl, n)
> + elif args.phase == 'delete-routes':
> + delete_routes(idl, n)
> + elif args.phase == 'delete-lbs':
> + delete_lbs(idl, n)
> + elif args.phase == 'delete-acls':
> + delete_acls(idl, n)
> + elif args.phase == 'delete-policies':
> + delete_policies(idl, n)
> + elif args.phase == 'delete-qos':
> + delete_qos(idl, n)
> +
> +
> +if __name__ == '__main__':
> + try:
> + main()
> + except error.Error as e:
> + sys.stderr.write(f'{e}\n')
> + sys.exit(1)
> diff --git a/tutorial/ovn-benchmark.sh b/tutorial/ovn-benchmark.sh
> new file mode 100755
> index 000000000..e68ef6a4e
> --- /dev/null
> +++ b/tutorial/ovn-benchmark.sh
> @@ -0,0 +1,561 @@
> +#!/bin/bash
> +
> +DEFAULT_NODES=200
> +
> +PROCESS_NAME=()
> +FILE_NAME=""
> +NODES=""
> +PROCESS_PIDS=()
> +FINAL_PEAK_KB=()
> +DEBUG=false
> +BATCH_SIZE=""
> +VALGRIND_MODE=false
> +VALGRIND_PIDS=()
> +MASSIF_FILES=()
> +BENCHMARK_TMPDIR=""
> +
> +on_interrupt() {
> + echo ""
> + if [ "$VALGRIND_MODE" = true ]; then
> + echo "Exiting benchmark. Restarting tracked sandbox processes..."
> + fi
> + exit 1
> +}
> +trap on_interrupt INT
> +
> +# In valgrind mode, the original daemons are killed and replaced with
> +# valgrind-wrapped copies. This cleanup ensures they are restored on
> +# both normal exit and Ctrl+C so the sandbox is not left broken.
> +cleanup() {
> + # Stop any valgrind processes still running.
> + for vpid in "${VALGRIND_PIDS[@]}"; do
> + kill "$vpid" 2>/dev/null
> + done
> +
> + if [ -n "$BENCHMARK_TMPDIR" ]; then
> + deadline=$(($(date +%s) + 30))
> + for vpid in "${VALGRIND_PIDS[@]}"; do
> + while [ -e "/proc/$vpid" ]; do
> + if [ "$(date +%s)" -ge "$deadline" ]; then
> + echo "Warning: valgrind PID $vpid" \
> + "did not exit; killing"
> + kill -9 "$vpid" 2>/dev/null
Nit: They're the same thing but I prefer "kill -KILL".
> + break
> + fi
> + sleep 0.1
> + done
> + done
> +
> + # Restart each daemon from the cmdline/cwd saved before we
> + # killed it, but only if it is not already running.
> + for i in "${!PROCESS_NAME[@]}"; do
> + pn="${PROCESS_NAME[$i]}"
> + if [ -f "$BENCHMARK_TMPDIR/cmdline.$pn" ] \
> + && ! pgrep -x "$pn" >/dev/null 2>&1; then
> + restart_cwd=$(cat "$BENCHMARK_TMPDIR/cwd.$pn")
> + mapfile -d '' restart_args < "$BENCHMARK_TMPDIR/cmdline.$pn"
> + # Drop trailing empty element from /proc/*/cmdline's final
> NUL.
> + if [ ${#restart_args[@]} -gt 0 ] \
> + && [ -z "${restart_args[-1]}" ]; then
> + unset 'restart_args[-1]'
> + fi
> + (cd "$restart_cwd" && "${restart_args[@]}" >/dev/null 2>&1)
> + fi
> + done
> +
> + rm -rf "$BENCHMARK_TMPDIR"
> + fi
> +}
> +trap cleanup EXIT
> +
> +while [[ $# -gt 0 ]]; do
> + case "$1" in
> + -h|--help|--usage)
> + echo "Usage: $0 [OPTIONS] [NODES] [PROCESS...]"
> + echo ""
> + echo "Arguments:"
> + echo " NODES Number of nodes to create" \
> + "(default: $DEFAULT_NODES)"
> + echo " PROCESS Process(es) to track:" \
> + "ovn-northd, ovn-controller"
> + echo " (default: both)"
> + echo ""
> + echo "Options:"
> + echo " -f, --file FILE Load NB database from file"
> + echo " -b, --batch-size N Nodes per chassis" \
> + "(default: NODES/10)"
> + echo " -v, --valgrind Track heap with Valgrind" \
> + "Massif (slower, accurate)"
> + echo " -d, --debug Enable debug output"
> + echo " -h, --help Show this help message"
> + echo ""
> + echo "Memory tracking:"
> + echo " Default: peak virtual memory (VmPeak) from"
> + echo " /proc/<pid>/status."
> + echo " --valgrind: peak heap via Valgrind Massif."
> + echo " Expect 10-50x slowdown; 500 nodes or fewer"
> + echo " recommended."
> + echo ""
> + echo "Examples:"
> + echo " $0 # 200 nodes, track both
> processes"
> + echo " $0 50 # 50 nodes"
> + echo " $0 50 ovn-northd # 50 nodes, track only
> ovn-northd"
> + echo " $0 --valgrind # Use Valgrind Massif, not
> VmPeak"
> + echo " $0 --file ovnnb_db.db # Load from file"
> + echo " $0 --debug 20 # 20 nodes with debug output"
> + echo " $0 50 -b 10 # 50 nodes, 10 per chassis"
> + echo ""
> + echo "Note: if only tracking one process, # of nodes is required"
> + exit 0
> + ;;
> + -b|--batch-size)
> + if [ -z "$2" ]; then
> + echo "Error: $1 requires an argument"
> + exit 1
> + fi
> + BATCH_SIZE="$2"
> + shift 2
> + ;;
> + -v|--valgrind)
> + VALGRIND_MODE=true
> + shift
> + ;;
> + -d|--debug)
> + DEBUG=true
> + shift
> + ;;
> + -f|--file)
> + if [ -z "$2" ]; then
> + echo "Error: $1 requires an argument"
> + exit 1
> + fi
> + FILE_NAME="$2"
> + shift 2
> + ;;
> + -*)
> + echo "Unknown option: $1"
> + exit 1
> + ;;
> + *)
> + if [ -z "$NODES" ]; then
> + NODES="$1"
> + else
> + # Normalize process names: accept both "northd" and
> + # "ovn-northd".
> + case "$1" in
> + northd)
> + PROCESS_NAME+=("ovn-northd")
> + ;;
> + controller)
> + PROCESS_NAME+=("ovn-controller")
> + ;;
> + *)
> + PROCESS_NAME+=("$1")
> + ;;
> + esac
> + fi
> + shift
> + ;;
> + esac
> +done
> +
> +# Must be run from inside the sandbox (tutorial/sandbox/).
> +if [ ! -d "$PWD/sandbox" ]; then
> + echo "Error: must be run from inside the OVN sandbox."
> + echo "Start one with: make sandbox"
> + exit 1
> +fi
> +
> +# Apply defaults if not set by user.
> +NODES=${NODES:-$DEFAULT_NODES}
> +
> +if [ -z "$BATCH_SIZE" ]; then
> + BATCH_SIZE=$((NODES / 10))
> +fi
> +if [ "$BATCH_SIZE" -lt 1 ]; then
> + BATCH_SIZE=1
> +fi
> +
> +# Track both processes if not specified.
> +if [ ${#PROCESS_NAME[@]} -eq 0 ]; then
> + PROCESS_NAME=("ovn-controller" "ovn-northd")
> +fi
> +
> +if [ "$DEBUG" = true ]; then
> + echo "Nodes: $NODES"
> + echo "Batch size: $BATCH_SIZE"
> + echo "Processes: ${PROCESS_NAME[*]}"
> + echo "File: ${FILE_NAME:-None}"
> +fi
> +
> +PID_ERRORS=false
> +for pn in "${PROCESS_NAME[@]}"; do
> + all_pids=$(pgrep -x "$pn")
> + if [ -z "$all_pids" ]; then
> + echo "Error: Could not find process matching '$pn'"
> + PID_ERRORS=true
> + continue
> + fi
> +
> + if [ "$(echo "$all_pids" | wc -l)" -gt 1 ]; then
> + echo "Error: Multiple $pn processes found" \
> + "(PIDs: $(echo $all_pids | tr '\n' ' '))"
> + PID_ERRORS=true
> + continue
> + fi
> +
> + PROCESS_PIDS+=("$all_pids")
> +done
> +
> +if [ "$PID_ERRORS" = true ]; then
> + echo "Kill stale processes or ensure only one sandbox is running."
> + exit 1
> +fi
> +
> +if [ "$DEBUG" = true ]; then
> + for i in "${!PROCESS_NAME[@]}"; do
> + echo "Tracking memory for ${PROCESS_NAME[$i]}" \
> + "(PID: ${PROCESS_PIDS[$i]})"
> + done
> +fi
> +
> +if [ "$VALGRIND_MODE" = true ]; then
> + if ! command -v valgrind >/dev/null 2>&1; then
> + echo "Error: valgrind is not installed or not in PATH"
> + exit 1
> + fi
> +
> + if [ "$NODES" -gt 500 ]; then
> + echo "Warning: valgrind mode with $NODES nodes may be very slow." \
> + "Consider using 500 or fewer nodes for heap profiling."
> + fi
> +
> + BENCHMARK_TMPDIR=$(mktemp -d)
> + echo "Restarting processes under Valgrind Massif..."
> + echo "(Expect 10-50x slowdown)"
> +
> + for i in "${!PROCESS_NAME[@]}"; do
> + pn="${PROCESS_NAME[$i]}"
> + pid="${PROCESS_PIDS[$i]}"
> +
> + # Read the original command line so we can restart the daemon later.
> + # /proc/<pid>/cmdline is NUL-separated; the trailing NUL produces an
> + # empty final element which must be dropped.
> + mapfile -d '' orig_args < /proc/$pid/cmdline
> + if [ ${#orig_args[@]} -gt 0 ] \
> + && [ -z "${orig_args[-1]}" ]; then
> + unset 'orig_args[-1]'
> + fi
> +
> + proc_cwd=$(readlink /proc/$pid/cwd)
> + massif_file="$BENCHMARK_TMPDIR/massif.$pn.out"
> + MASSIF_FILES+=("$massif_file")
> +
> + # Save original cmdline and cwd so we can restart after valgrind.
> + cp /proc/$pid/cmdline "$BENCHMARK_TMPDIR/cmdline.$pn"
> + echo "$proc_cwd" > "$BENCHMARK_TMPDIR/cwd.$pn"
> +
> + # Valgrind runs the process in the foreground, so strip args
> + # that assume daemonized execution:
> + # --detach : can't daemonize under valgrind
> + # --no-chdir : only meaningful with --detach
> + # --pidfile : valgrind doesn't write one (OVS uses
> + # optional_argument so bare --pidfile never
> + # takes a separate arg; only --pidfile=FILE)
> + # --monitor : would fork a monitor child under valgrind,
> + # causing two processes to race on massif output
> + filtered_args=()
> + for arg in "${orig_args[@]}"; do
> + case "$arg" in
> + --detach|--no-chdir|--monitor) ;;
> + --pidfile|--pid-file) ;;
> + --pidfile=*|--pid-file=*) ;;
> + *) filtered_args+=("$arg") ;;
> + esac
> + done
> +
> + if [ "$DEBUG" = true ]; then
> + echo "Stopping $pn (PID $pid)..."
> + fi
> +
> + kill "$pid"
> + while [ -e "/proc/$pid" ]; do sleep 0.1; done
This may end up looping indefinitely if for some reason "$pid" refuses
to terminate, e.g., ignores SIGTERM.
I think we'll need a sort of "kill_insist()" or similar function that
would force a SIGKILL to be sent if the process doesn't end in time, e.g.:
kill_insist() {
local pid=$1
local timeout_ms=${2:-30000}
env kill -0 "$pid" 2> /dev/null && \
env kill --timeout "$timeout_ms" KILL "$pid"
}
We could use this function inside cleanup() too where we're currently
cleaning up any leftover valgrind processes.
We need "env" because the bash builtin kill doesn't support --timeout.
The rest looks good to me so, aside from the load balancer group comment
and the check/note about valgrind not working with ubsan/asan (discussed
in the other thread) which can be potential follow ups, I'm considering
applying this patch to main with the following incremental change
squashed in. Please let me know what you think about it.
Thanks,
Dumitru
diff --git a/tutorial/ovn-benchmark.sh b/tutorial/ovn-benchmark.sh
index e68ef6a4e6..01c090e846 100755
--- a/tutorial/ovn-benchmark.sh
+++ b/tutorial/ovn-benchmark.sh
@@ -23,29 +23,24 @@ on_interrupt() {
}
trap on_interrupt INT
+kill_insist() {
+ local pid=$1
+ local timeout_ms=${2:-30000}
+
+ env kill -0 "$pid" 2> /dev/null && \
+ env kill --timeout "$timeout_ms" KILL "$pid"
+}
+
# In valgrind mode, the original daemons are killed and replaced with
# valgrind-wrapped copies. This cleanup ensures they are restored on
# both normal exit and Ctrl+C so the sandbox is not left broken.
cleanup() {
# Stop any valgrind processes still running.
for vpid in "${VALGRIND_PIDS[@]}"; do
- kill "$vpid" 2>/dev/null
+ kill_insist $vpid
done
if [ -n "$BENCHMARK_TMPDIR" ]; then
- deadline=$(($(date +%s) + 30))
- for vpid in "${VALGRIND_PIDS[@]}"; do
- while [ -e "/proc/$vpid" ]; do
- if [ "$(date +%s)" -ge "$deadline" ]; then
- echo "Warning: valgrind PID $vpid" \
- "did not exit; killing"
- kill -9 "$vpid" 2>/dev/null
- break
- fi
- sleep 0.1
- done
- done
-
# Restart each daemon from the cmdline/cwd saved before we
# killed it, but only if it is not already running.
for i in "${!PROCESS_NAME[@]}"; do
@@ -278,8 +273,7 @@ if [ "$VALGRIND_MODE" = true ]; then
echo "Stopping $pn (PID $pid)..."
fi
- kill "$pid"
- while [ -e "/proc/$pid" ]; do sleep 0.1; done
+ kill_insist $pid
if [ "$DEBUG" = true ]; then
echo "Starting $pn under valgrind:"
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
