On Wed, Feb 17, 2021, at 17:34, Gaetan Rivet wrote:
> Conntracks are executed within the datapath. Locks along this path are crucial
> and their critical section should be minimal. The global 'ct_lock' is
> necessary
> before any action taken on connection states. This lock is needed for many
> operations on the conntrack, slowing down the datapath.
>
> The cleanup thread 'ct_clean' will take it to do its job. As it can hold it a
> long time, the thread is limited in amount of connection cleaned per round,
> and calls are rate-limited.
>
> * Timeout policies locking is contrived to avoid deadlock.
> Anytime a connection state is updated, during its update it is unlocked,
> 'ct_lock' is taken, then the connection is locked again. Then the reverse
> is done for unlock.
>
> * Scalability is poor. The global ct_lock needs to be taken before applying
> any change to a conn object. This is backward: local changes to smaller
> objects should be independent, then the global lock should only be taken
> once
> the rest of the work is done, the goal being to have the smallest possible
> critical section.
>
> It can be improved. Using RCU-friendly structures for connections, zone limits
> and timeout policies, read-first workload is improved and the precedence of
> the
> global 'ct_lock' and local 'conn->lock' can be inversed.
>
> Running the conntrack benchmark we see these changes:
> ./tests/ovstest test-conntrack benchmark 300 32
>
> code \ N 1 2 4 8
> Before 2310 2766 6117 19838 (ms)
>After 2072 2084 2653 4541 (ms)
>
> One thread in the benchmark executes the task of a PMD, while the
> 'ct_clean' thread
> runs in background as well.
>
> Github actions: https://github.com/grivet/ovs/actions/runs/574446345
>
> Gaetan Rivet (9):
> conntrack: Use rcu-lists to store conn expirations
> conntrack: Use a cmap to store zone limits
> conntrack: Init hash basis first at creation
> conntrack-tp: Use a cmap to store timeout policies
> conntrack: Inverse conn and ct lock precedence
> conntrack: Do not schedule zero ms timers
> conntrack: Do not rate limit ct-sweep
> conntrack: Do not log empty ct-sweep
> conntrack: Use an atomic conn expiration value
>
> lib/conntrack-private.h | 96 +--
> lib/conntrack-tp.c | 137 +++---
> lib/conntrack.c | 206 +---
> lib/conntrack.h | 4 +-
> lib/dpif-netdev.c | 5 +-
> 5 files changed, 280 insertions(+), 168 deletions(-)
>
> --
> 2.30.0
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
The v2 of this series has been sent:
https://patchwork.ozlabs.org/project/openvswitch/list/?series=240241
--
Gaetan Rivet
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev