Re: [ovs-dev] [PATCH v4] dpdk: Allow retaining CAP_SYS_RAWIO privileges

2023-03-16 Thread Gaetan Rivet via dev
> From: Aaron Conole > Date: Thursday, 16 March 2023 at 13:00 > To: d...@openvswitch.org > Cc: Robin Jarry , Gaetan Rivet , Ilya > Maximets , Eli Britstein , Maxime > Coquelin , Jason Gunthorpe , > Majd Dibbiny , David Marchand march...@redhat.com>, Simon Ho

Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges

2023-02-23 Thread Gaetan Rivet via dev
> -Original Message- > From: Aaron Conole mailto:acon...@redhat.com>> > Date: Wednesday 22 February 2023 at 18:11 > To: "d...@openvswitch.org <mailto:d...@openvswitch.org>" > mailto:d...@openvswitch.org>> > Cc: Eli Britstein mailto:e

Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges

2023-02-23 Thread Gaetan Rivet via dev
> -Original Message- > From: Robin Jarry mailto:rja...@redhat.com>> > Date: Thursday 23 February 2023 at 22:43 > To: Gaetan Rivet mailto:gaet...@nvidia.com>>, Aaron > Conole mailto:acon...@redhat.com>> > Cc: "d...@openvswitch.org &

Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges

2023-02-23 Thread Gaetan Rivet via dev
@openvswitch.org>>, Eli Britstein > mailto:el...@nvidia.com>>, Gaetan Rivet > mailto:gaet...@nvidia.com>>, Ilya Maximets > >, Maxime Coquelin > mailto:maxime.coque...@redhat.com>>, Jason > Gunthorpe mailto:j...@nvidia.com>>, Majd Dibbiny > mailto:m...@nvid

Re: [ovs-dev] [PATCH 1/1] daemon-unix: Support OVS-DPDK HW offloads for non-root user

2023-02-09 Thread Gaetan Rivet via dev
>-Original Message- >From: dev <mailto:ovs-dev-boun...@openvswitch.org>> on behalf of Eelco Chaudron >mailto:echau...@redhat.com>> >Date: Monday 23 January 2023 at 10:44 >To: Gaetan Rivet mailto:gaet...@nvidia.com>> >Cc: ovs dev mailto:d...@openv

[ovs-dev] [PATCH v4 5/5] conntrack: Use an atomic conn expiration value

2022-03-25 Thread Gaetan Rivet
are not undefined behaviour. Reading an atomic is however less costly than taking and releasing a lock. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 2 +- lib/conntrack.c | 27

[ovs-dev] [PATCH v4 4/5] conntrack: Inverse conn and ct lock precedence

2022-03-25 Thread Gaetan Rivet
ant structures. This will reduce contention on 'ct_lock', which impairs scalability when the connection tracker is used by many threads. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/conntrack-private.h | 7 -- lib/conntrack-tp.c | 30 +-

[ovs-dev] [PATCH v4 3/5] conntrack-tp: Use a cmap to store timeout policies

2022-03-25 Thread Gaetan Rivet
connection insertion path. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 54 +++-- lib/conntrack.c | 9 --- lib/conntrack.h | 2 +- 4 files c

[ovs-dev] [PATCH v4 2/5] conntrack: Use a cmap to store zone limits

2022-03-25 Thread Gaetan Rivet
s reducing contention. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/conntrack-private.h | 2 +- lib/conntrack.c | 70 - lib/conntrack.h | 2 +- lib/dpif-netdev.c | 5 +-- 4 files changed, 53 insertions(+), 26

[ovs-dev] [PATCH v4 0/5] conntrack: improve multithread scalability

2022-03-25 Thread Gaetan Rivet
etracked. Paolo's alternative series [2] can also improve the same metric. I am not sure which one would be best between the two, I am sending this revised version so that it is available for public comment. [1]: https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385470.html [2]: https:

[ovs-dev] [PATCH v4 1/5] conntrack: Use mpsc-queue to store conn expirations

2022-03-25 Thread Gaetan Rivet
the queue. Once the locks have been reworked, it means neither the dataplane threads nor 'ct_clean' have to take a lock to update the expiration lists (assuming the consumer lock is perpetually held by 'ct_clean'); Signed-off-by: Gaetan Rivet Reviewed-by: Eli Brits

[ovs-dev] [PATCH] ofproto: Use xlate map for uuid lookups

2022-02-23 Thread Gaetan Rivet
port' to user action cookie.") Suggested-by: Adrian Moreno Signed-off-by: Yunjian Wang Signed-off-by: Gaetan Rivet --- Following the discussion on the fix https://patchwork.ozlabs.org/project/openvswitch/patch/1638530715-44436-1-git-send-email-wangyunj...@huawei.com/ I tested it wit

[ovs-dev] [PATCH v1 1/3] dpif-netdev: Move port flush after datapath reconfiguration

2022-02-04 Thread Gaetan Rivet
from offload thread.") Fixes: 62d1c28e9ce0 ("dpif-netdev: Flush offload rules upon port deletion.") Signed-off-by: Ilya Maximets Signed-off-by: Gaetan Rivet --- lib/dpif-netdev.c | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netdev.c b/lib

[ovs-dev] [PATCH v1 3/3] dpif-netdev: Use dp_netdev reference in offload threads

2022-02-04 Thread Gaetan Rivet
datapath deletion. No further offload request should be found past a flush, so it is safe to keep this reference in the offload item. Signed-off-by: Gaetan Rivet --- lib/dpif-netdev.c | 50 +++ 1 file changed, 24 insertions(+), 26 deletions(-) diff

[ovs-dev] [PATCH v1 2/3] dpif-netdev: Fix a race condition in deletion of offloaded flows

2022-02-04 Thread Gaetan Rivet
flow. This problem has been observed while adding and deleting flows in a loop. To fix this, always enqueue flow deletion regardless of the flow->mark being set. Fixes: 241bad15d99a("dpif-netdev: associate flow with a mark id") Signed-off-by: Sriharsha Basavapatna Signed-off-by:

[ovs-dev] [PATCH v1 0/3] Fix offload rule flush race condition

2022-02-04 Thread Gaetan Rivet
excruciatingly slow and could not progress much. It reached datapath deletion without panicking and no crash was seen, even though I had to interrupt the test after a few hours. Gaetan Rivet (2): dpif-netdev: Move port flush after datapath reconfiguration dpif-netdev: Use dp_netdev reference in offload

[ovs-dev] [PATCH v5 27/27] netdev-dpdk: Remove rte-flow API access locks

2021-09-08 Thread Gaetan Rivet
The rte_flow DPDK API was made thread-safe [1] in release 20.11. Now that the DPDK offload provider in OVS is thread safe, remove the locks. [1]: http://mails.dpdk.org/archives/dev/2020-October/184251.html Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin

[ovs-dev] [PATCH v5 25/27] dpif-netdev: Replace port mutex by rwlock

2021-09-08 Thread Gaetan Rivet
5c2645 ("dpif-netdev: Make datapath port mutex recursive.") [2]: 12d0edd75eba ("dpif-netdev: Avoid deadlock with offloading during PMD thread deletion."). Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c

[ovs-dev] [PATCH v5 21/27] netdev-offload-dpdk: Lock rte_flow map access

2021-09-08 Thread Gaetan Rivet
. The lock is still needed to protect against changes to netdev port mapping. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 8 ++--- lib/netdev-offload-dpdk.c | 61 --- 2 files changed, 61

[ovs-dev] [PATCH v5 23/27] dpif-netdev: Use lockless queue to manage offloads

2021-09-08 Thread Gaetan Rivet
of some latency to manage offloads after an inactivity period. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 109 -- 1 file changed, 57 insertions(+), 52 deletions(-) diff --git a/lib

[ovs-dev] [PATCH v5 26/27] dpif-netdev: Use one or more offload threads

2021-09-08 Thread Gaetan Rivet
multiplied by the number of requested threads and used separately. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 290 -- lib/netdev-offload-dpdk.c | 7 +- 2 files changed, 193 insertions

[ovs-dev] [PATCH v5 24/27] dpif-netdev: Make megaflow and mark mappings thread objects

2021-09-08 Thread Gaetan Rivet
the offload threads. To prepare this change, move the mappings within the offload thread structure. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 45 + 1 file changed, 21 insertions(+), 24 d

[ovs-dev] [PATCH v5 20/27] netdev-offload-dpdk: Use per-thread HW offload stats

2021-09-08 Thread Gaetan Rivet
The implementation of hardware offload counters in currently meant to be managed by a single thread. Use the offload thread pool API to manage one counter per thread. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 16

[ovs-dev] [PATCH v5 22/27] netdev-offload-dpdk: Protect concurrent offload destroy/query

2021-09-08 Thread Gaetan Rivet
locks on the rte_flow API, mutually exclude offload query and deletion from concurrent execution. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 39 --- 1 file changed, 36 insertions(+), 3

[ovs-dev] [PATCH v5 19/27] dpif-netdev: Execute flush from offload thread

2021-09-08 Thread Gaetan Rivet
introducing parallel offloads, execute the flush operation in the offload thread. Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 126 -- 1 file changed, 122 insertions(+), 4 deletions(-) diff --git a/lib/dpif-netdev.c b/lib

[ovs-dev] [PATCH v5 16/27] dpif-netdev: Postpone flow offload item freeing

2021-09-08 Thread Gaetan Rivet
messages freeing using the RCU. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index c4672e6e5..c3d211858 100644 --- a/lib

[ovs-dev] [PATCH v5 18/27] dpif-netdev: Introduce tagged union of offload requests

2021-09-08 Thread Gaetan Rivet
ot of memory. Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 137 -- 1 file changed, 96 insertions(+), 41 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 24ecaa0a8..e0052a65b 100644 --- a/lib

[ovs-dev] [PATCH v5 17/27] dpif-netdev: Use id-fpool for mark allocation

2021-09-08 Thread Gaetan Rivet
Use the netdev-offload multithread API to allow multiple thread allocating marks concurrently. Initialize only once the pool in a multithread context by using the ovsthread_once type. Use the id-fpool module for faster concurrent ID allocation. Signed-off-by: Gaetan Rivet Reviewed-by: Eli

[ovs-dev] [PATCH v5 15/27] dpif-netdev: Quiesce offload thread periodically

2021-09-08 Thread Gaetan Rivet
. Instead attempt to quiesce every 10 ms at most. While the queue is empty, the offload thread remains quiescent. [1]: 81ac8b3b194c ("dpif-netdev: Do RCU synchronization at fixed interval in PMD main loop.") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxim

[ovs-dev] [PATCH v5 12/27] mpsc-queue: Module for lock-free message passing

2021-09-08 Thread Gaetan Rivet
-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/automake.mk | 2 + lib/mpsc-queue.c| 251 + lib/mpsc-queue.h| 190 ++ tests/automake.mk | 1 + tests/library.at| 5 + tests/test-mpsc-queue.c | 772

[ovs-dev] [PATCH v5 13/27] id-fpool: Module for fast ID generation

2021-09-08 Thread Gaetan Rivet
Avg id-fpool new: 9 11 11 10 10 ms id-fpool del: 5 6 6 5 5 ms id-fpool mix: 16 16 16 16 16 ms id-fpool rnd: 20 20 20 20 20 ms Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin

[ovs-dev] [PATCH v5 11/27] ovs-atomic: Expose atomic exchange operation

2021-09-08 Thread Gaetan Rivet
The atomic exchange operation is a useful primitive that should be available as well. Most compilers already expose or offer a way to use it, but a single symbol needs to be defined. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/ovs-atomic-c++.h

[ovs-dev] [PATCH v5 14/27] netdev-offload: Add multi-thread API

2021-09-08 Thread Gaetan Rivet
reclamation purposes. In such case, it will get a default offload thread ID of 0. Care must be taken that using this thread ID is safe concurrently with the offload threads. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-provider.h | 1

[ovs-dev] [PATCH v5 09/27] mov-avg: Add a moving average helper structure

2021-09-08 Thread Gaetan Rivet
Add a new module offering a helper to compute the Cumulative Moving Average (CMA) and the Exponential Moving Average (EMA) of a series of values. Use the new helpers to add latency metrics in dpif-netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin

[ovs-dev] [PATCH v5 10/27] dpif-netdev: Implement hardware offloads stats query

2021-09-08 Thread Gaetan Rivet
In the netdev datapath, keep track of the enqueued offloads between the PMDs and the offload thread. Additionally, query each netdev for their hardware offload counters. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 90

[ovs-dev] [PATCH v5 07/27] dpctl: Add function to read hardware offload statistics

2021-09-08 Thread Gaetan Rivet
its handling of hardware offloads. Call the new API from dpctl. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpctl.c | 36 lib/dpif-netdev.c | 1 + lib/dpif-netlink.c | 1 + lib/dpif-provider.h

[ovs-dev] [PATCH v5 08/27] dpif-netdev: Rename offload thread structure

2021-09-08 Thread Gaetan Rivet
The offload management in userspace is done through a separate thread. The naming of the structure holding the objects used for synchronization with the dataplane is generic and nondescript. Clarify the object function by renaming it. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein

[ovs-dev] [PATCH v5 06/27] netdev-offload-dpdk: Implement hw-offload statistics read

2021-09-08 Thread Gaetan Rivet
In the DPDK offload provider, keep track of inserted rte_flow and report it when queried. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/lib

[ovs-dev] [PATCH v5 05/27] netdev-offload-dpdk: Use per-netdev offload metadata

2021-09-08 Thread Gaetan Rivet
deletion. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 126 +- lib/netdev-offload.h | 2 + 2 files changed, 113 insertions(+), 15 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b

[ovs-dev] [PATCH v5 04/27] netdev: Add flow API uninit function

2021-09-08 Thread Gaetan Rivet
Add a new operation for flow API providers to uninitialize when the API is disassociated from a netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-provider.h | 3 +++ lib/netdev-offload.c | 4 2 files changed, 7

[ovs-dev] [PATCH v5 03/27] tests: Add ovs-barrier unit test

2021-09-08 Thread Gaetan Rivet
No unit test exist currently for the ovs-barrier type. It is however crucial as a building block and should be verified to work as expected. Create a simple test verifying the basic function of ovs-barrier. Integrate the test as part of the test suite. Signed-off-by: Gaetan Rivet Reviewed-by

[ovs-dev] [PATCH v5 02/27] dpif-netdev: Rename flow offload thread

2021-09-08 Thread Gaetan Rivet
ovs_strlcpy silently fails to copy the thread name if it is too long. Rename the flow offload thread to differentiate it from the main thread. Fixes: 02bb2824e51d ("dpif-netdev: do hw flow offload in a thread") Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-ne

[ovs-dev] [PATCH v5 01/27] ovs-thread: Fix barrier use-after-free

2021-09-08 Thread Gaetan Rivet
immediately after blocking on it. Fixes: d8043da7182a ("ovs-thread: Implement OVS specific barrier.") Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/ovs-thread.c | 61 +++- lib/ovs-thread.h | 6 ++--- 2 files changed, 53

[ovs-dev] [PATCH v5 00/27] dpif-netdev: Parallel offload processing

2021-09-08 Thread Gaetan Rivet
s. CI result: https://github.com/grivet/ovs/actions/runs/1212804378 Gaetan Rivet (27): ovs-thread: Fix barrier use-after-free dpif-netdev: Rename flow offload thread tests: Add ovs-barrier unit test netdev: Add flow API uninit function netdev-offload-dpdk: Use per-netdev offload metadata

[ovs-dev] [PATCH v3 8/8] conntrack: Use an atomic conn expiration value

2021-06-15 Thread Gaetan Rivet
are not undefined behaviour. Reading an atomic is however less costly than taking and releasing a lock. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 2 +- lib/conntrack.c | 27

[ovs-dev] [PATCH v3 7/8] conntrack: Inverse conn and ct lock precedence

2021-06-15 Thread Gaetan Rivet
ant structures. This will reduce contention on 'ct_lock', which impairs scalability when the connection tracker is used by many threads. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/conntrack-private.h | 7 -- lib/conntrack-tp.c | 30

[ovs-dev] [PATCH v3 6/8] conntrack-tp: Use a cmap to store timeout policies

2021-06-15 Thread Gaetan Rivet
connection insertion path. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 54 +++-- lib/conntrack.c | 9 --- lib/conntrack.h | 2 +- 4 files c

[ovs-dev] [PATCH v3 2/8] ovs-atomic: Expose atomic exchange operation

2021-06-15 Thread Gaetan Rivet
The atomic exchange operation is a useful primitive that should be available as well. Most compiler already expose or offer a way to use it, but a single symbol needs to be defined. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/ovs-atomic-c++.h | 3 +++ lib/ovs-atomic

[ovs-dev] [PATCH v3 5/8] conntrack: Use a cmap to store zone limits

2021-06-15 Thread Gaetan Rivet
s reducing contention. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/conntrack-private.h | 2 +- lib/conntrack.c | 70 - lib/conntrack.h | 2 +- lib/dpif-netdev.c | 5 +-- 4 files changed, 53 insertions(+), 26

[ovs-dev] [PATCH v3 3/8] mpsc-queue: Module for lock-free message passing

2021-06-15 Thread Gaetan Rivet
-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 2 + lib/mpsc-queue.c| 251 + lib/mpsc-queue.h| 190 ++ tests/automake.mk | 1 + tests/library.at| 5 + tests/test-mpsc-queue.c | 772

[ovs-dev] [PATCH v3 4/8] conntrack: Use mpsc-queue to store conn expirations

2021-06-15 Thread Gaetan Rivet
the queue. Once the locks have been reworked, it means neither the dataplane threads nor 'ct_clean' have to take a lock to update the expiration lists (assuming the consumer lock is perpetually held by 'ct_clean'); Signed-off-by: Gaetan Rivet Reviewed-by: Eli Brits

[ovs-dev] [PATCH v3 0/8] conntrack: improve multithread scalability

2021-06-15 Thread Gaetan Rivet
0 32 code \ N 1 2 4 8 Before598 1656 12612 39301 (ms) After293 337 427 893 (ms) I replicate the numbers on a 24-cores machine as well. The benchmark is not very accurate as no core pinning and no isolation is done. Gaetan Rivet (8): conntrack: Init hash basi

[ovs-dev] [PATCH v3 1/8] conntrack: Init hash basis first at creation

2021-06-15 Thread Gaetan Rivet
rrect hash value. Fixes: 2078901a4c14 ("userspace: Add conntrack timeout policy support.") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu --- lib/conntrack.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/conntrack.c b/l

[ovs-dev] [PATCH v4 24/27] dpif-netdev: Make megaflow and mark mappings thread objects

2021-06-09 Thread Gaetan Rivet
the offload threads. To prepare this change, move the mappings within the offload thread structure. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 47 ++- 1 file changed, 22 insertions(+), 25

[ovs-dev] [PATCH v4 23/27] dpif-netdev: Use lockless queue to manage offloads

2021-06-09 Thread Gaetan Rivet
of some latency to manage offloads after an inactivity period. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 109 -- 1 file changed, 57 insertions(+), 52 deletions(-) diff --git a/lib

[ovs-dev] [PATCH v4 27/27] netdev-dpdk: Remove rte-flow API access locks

2021-06-09 Thread Gaetan Rivet
The rte_flow DPDK API was made thread-safe [1] in release 20.11. Now that the DPDK offload provider in OVS is thread safe, remove the locks. [1]: http://mails.dpdk.org/archives/dev/2020-October/184251.html Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin

[ovs-dev] [PATCH v4 26/27] dpif-netdev: Use one or more offload threads

2021-06-09 Thread Gaetan Rivet
multiplied by the number of requested threads and used separately. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 304 +- lib/netdev-offload-dpdk.c | 7 +- 2 files changed, 204 insertions

[ovs-dev] [PATCH v4 25/27] dpif-netdev: Replace port mutex by rwlock

2021-06-09 Thread Gaetan Rivet
5c2645 ("dpif-netdev: Make datapath port mutex recursive.") [2]: 12d0edd75eba ("dpif-netdev: Avoid deadlock with offloading during PMD thread deletion."). Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c

[ovs-dev] [PATCH v4 22/27] netdev-offload-dpdk: Protect concurrent offload destroy/query

2021-06-09 Thread Gaetan Rivet
locks on the rte_flow API, mutually exclude offload query and deletion from concurrent execution. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 39 --- 1 file changed, 36 insertions(+), 3

[ovs-dev] [PATCH v4 21/27] netdev-offload-dpdk: Lock rte_flow map access

2021-06-09 Thread Gaetan Rivet
. The lock is still needed to protect against changes to netdev port mapping. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 8 ++--- lib/netdev-offload-dpdk.c | 61 --- 2 files changed, 61

[ovs-dev] [PATCH v4 20/27] netdev-offload-dpdk: Use per-thread HW offload stats

2021-06-09 Thread Gaetan Rivet
The implementation of hardware offload counters in currently meant to be managed by a single thread. Use the offload thread pool API to manage one counter per thread. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 16

[ovs-dev] [PATCH v4 19/27] dpif-netdev: Execute flush from offload thread

2021-06-09 Thread Gaetan Rivet
introducing parallel offloads, execute the flush operation in the offload thread. Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 126 -- 1 file changed, 122 insertions(+), 4 deletions(-) diff --git a/lib/dpif-netdev.c b/lib

[ovs-dev] [PATCH v4 17/27] dpif-netdev: Use id-fpool for mark allocation

2021-06-09 Thread Gaetan Rivet
Use the netdev-offload multithread API to allow multiple thread allocating marks concurrently. Initialize only once the pool in a multithread context by using the ovsthread_once type. Use the id-fpool module for faster concurrent ID allocation. Signed-off-by: Gaetan Rivet Reviewed-by: Eli

[ovs-dev] [PATCH v4 18/27] dpif-netdev: Introduce tagged union of offload requests

2021-06-09 Thread Gaetan Rivet
ot of memory. Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 128 -- 1 file changed, 89 insertions(+), 39 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index b8fd49f5d..1d7e55d47 100644 --- a/lib

[ovs-dev] [PATCH v4 16/27] dpif-netdev: Postpone flow offload item freeing

2021-06-09 Thread Gaetan Rivet
messages freeing using the RCU. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index e403e461a..75b289904 100644 --- a/lib

[ovs-dev] [PATCH v4 15/27] dpif-netdev: Quiesce offload thread periodically

2021-06-09 Thread Gaetan Rivet
. Instead attempt to quiesce every 10 ms at most. While the queue is empty, the offload thread remains quiescent. [1]: 81ac8b3b194c ("dpif-netdev: Do RCU synchronization at fixed interval in PMD main loop.") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-net

[ovs-dev] [PATCH v4 13/27] id-fpool: Module for fast ID generation

2021-06-09 Thread Gaetan Rivet
Avg id-fpool new: 9 11 11 10 10 ms id-fpool del: 5 6 6 5 5 ms id-fpool mix: 16 16 16 16 16 ms id-fpool rnd: 20 20 20 20 20 ms Signed-off-by: Gaetan Rivet --- lib/automake.mk | 2 + l

[ovs-dev] [PATCH v4 14/27] netdev-offload: Add multi-thread API

2021-06-09 Thread Gaetan Rivet
reclamation purposes. In such case, it will get a default offload thread ID of 0. Care must be taken that using this thread ID is safe concurrently with the offload threads. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-provider.h | 1

[ovs-dev] [PATCH v4 12/27] mpsc-queue: Module for lock-free message passing

2021-06-09 Thread Gaetan Rivet
-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 2 + lib/mpsc-queue.c| 251 + lib/mpsc-queue.h| 190 ++ tests/automake.mk | 1 + tests/library.at| 5 + tests/test-mpsc-queue.c | 772

[ovs-dev] [PATCH v4 11/27] ovs-atomic: Expose atomic exchange operation

2021-06-09 Thread Gaetan Rivet
The atomic exchange operation is a useful primitive that should be available as well. Most compiler already expose or offer a way to use it, but a single symbol needs to be defined. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/ovs-atomic-c++.h | 3 +++ lib/ovs-atomic

[ovs-dev] [PATCH v4 10/27] dpif-netdev: Implement hardware offloads stats query

2021-06-09 Thread Gaetan Rivet
In the netdev datapath, keep track of the enqueued offloads between the PMDs and the offload thread. Additionally, query each netdev for their hardware offload counters. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 90

[ovs-dev] [PATCH v4 09/27] mov-avg: Add a moving average helper structure

2021-06-09 Thread Gaetan Rivet
Add a new module offering a helper to compute the Cumulative Moving Average (CMA) and the Exponential Moving Average (EMA) of a series of values. Use the new helpers to add latency metrics in dpif-netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin

[ovs-dev] [PATCH v4 08/27] dpif-netdev: Rename offload thread structure

2021-06-09 Thread Gaetan Rivet
The offload management in userspace is done through a separate thread. The naming of the structure holding the objects used for synchronization with the dataplane is generic and nondescript. Clarify the object function by renaming it. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein

[ovs-dev] [PATCH v4 07/27] dpctl: Add function to read hardware offload statistics

2021-06-09 Thread Gaetan Rivet
its handling of hardware offloads. Call the new API from dpctl. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpctl.c | 36 lib/dpif-netdev.c | 1 + lib/dpif-netlink.c | 1 + lib/dpif-provider.h

[ovs-dev] [PATCH v4 05/27] netdev-offload-dpdk: Use per-netdev offload metadata

2021-06-09 Thread Gaetan Rivet
deletion. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 131 -- lib/netdev-offload.h | 2 + 2 files changed, 114 insertions(+), 19 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b

[ovs-dev] [PATCH v4 04/27] netdev: Add flow API uninit function

2021-06-09 Thread Gaetan Rivet
Add a new operation for flow API providers to uninitialize when the API is disassociated from a netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-provider.h | 3 +++ lib/netdev-offload.c | 4 2 files changed, 7

[ovs-dev] [PATCH v4 06/27] netdev-offload-dpdk: Implement hw-offload statistics read

2021-06-09 Thread Gaetan Rivet
In the DPDK offload provider, keep track of inserted rte_flow and report it when queried. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/netdev-offload-dpdk.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/lib

[ovs-dev] [PATCH v4 02/27] dpif-netdev: Rename flow offload thread

2021-06-09 Thread Gaetan Rivet
ovs_strlcpy silently fails to copy the thread name if it is too long. Rename the flow offload thread to differentiate it from the main thread. Fixes: 02bb2824e51d ("dpif-netdev: do hw flow offload in a thread") Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/dpif-ne

[ovs-dev] [PATCH v4 03/27] tests: Add ovs-barrier unit test

2021-06-09 Thread Gaetan Rivet
No unit test exist currently for the ovs-barrier type. It is however crucial as a building block and should be verified to work as expected. Create a simple test verifying the basic function of ovs-barrier. Integrate the test as part of the test suite. Signed-off-by: Gaetan Rivet Reviewed-by

[ovs-dev] [PATCH v4 01/27] ovs-thread: Fix barrier use-after-free

2021-06-09 Thread Gaetan Rivet
immediately after blocking on it. Fixes: d8043da7182a ("ovs-thread: Implement OVS specific barrier.") Signed-off-by: Gaetan Rivet Reviewed-by: Maxime Coquelin --- lib/ovs-thread.c | 61 +++- lib/ovs-thread.h | 6 ++--- 2 files changed, 53

[ovs-dev] [PATCH v4 00/27] dpif-netdev: Parallel offload processing

2021-06-09 Thread Gaetan Rivet
ueue rnd: avg 97.2 | stdev 15.7 | max 130 | min64 4 threads: -- id-fpool rnd: avg 25.6 | stdev2.4 | max28 | min20 seq-pool rnd: avg 89.3 | stdev9.7 | max 101 | min 65 id-queue rnd: avg 347.9 | stdev 44.1 | max 410 | min 236 Gaetan Rivet (27): ovs

[ovs-dev] [PATCH v2 1/8] configure: add --enable-asan option

2021-05-20 Thread Gaetan Rivet
Add a configure option to enable ASAN in a simple way. Adding an AC variable to allow checking for support in the testsuite. Signed-off-by: Gaetan Rivet --- .ci/linux-build.sh | 4 ++-- NEWS | 1 + acinclude.m4 | 16 configure.ac | 1 + tests

[ovs-dev] [PATCH v2 8/8] ovs-rcu: Add blocking RCU mode

2021-05-20 Thread Gaetan Rivet
-rcu-uaf quiesce-start-end # The testsuite can be used as well make check TESTSUITEFLAGS='-k rcu' Signed-off-by: Gaetan Rivet --- .ci/linux-build.sh | 4 ++ .github/workflows/build-and-test.yml | 7 +++ NEWS |

[ovs-dev] [PATCH v2 0/8] RCU: Add blocking mode for debugging

2021-05-20 Thread Gaetan Rivet
: * Rebased on master * Added documentation in lib/ovs-rcu.h following Ben's suggestion. CI: https://github.com/grivet/ovs/actions/runs/860557554 Gaetan Rivet (8): configure: add --enable-asan option tests: Add ovs-barrier unit test tests: Add RCU postpone test tests: Add ASAN use-

[ovs-dev] [PATCH v2 4/8] tests: Add ASAN use-after-free validation with RCU

2021-05-20 Thread Gaetan Rivet
-off-by: Gaetan Rivet --- tests/automake.mk| 1 + tests/library.at | 33 +++ tests/test-rcu-uaf.c | 98 3 files changed, 132 insertions(+) create mode 100644 tests/test-rcu-uaf.c diff --git a/tests/automake.mk b/tests/automake.mk

[ovs-dev] [PATCH v2 7/8] ovs-rcu: Remove unused perthread mutex

2021-05-20 Thread Gaetan Rivet
A mutex is allocated, initialized and destroyed, without being used in the perthread structure. Signed-off-by: Gaetan Rivet --- lib/ovs-rcu.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/lib/ovs-rcu.c b/lib/ovs-rcu.c index cde1e925b..1866bd308 100644 --- a/lib/ovs-rcu.c +++ b/lib/ovs

[ovs-dev] [PATCH v2 6/8] ovs-thread: Quiesce when joining pthreads

2021-05-20 Thread Gaetan Rivet
Joining pthreads makes the caller quiescent. It should register as such, as joined threads may wait on an RCU callback executing before quitting, deadlocking the caller. Signed-off-by: Gaetan Rivet --- lib/ovs-thread.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deletions

[ovs-dev] [PATCH v2 3/8] tests: Add RCU postpone test

2021-05-20 Thread Gaetan Rivet
Add a simple postponing test verifying RCU callbacks have executed and RCU exits in order. Add as part of library unit-tests. Signed-off-by: Gaetan Rivet --- tests/library.at | 8 ++- tests/test-rcu.c | 61 2 files changed, 68 insertions

[ovs-dev] [PATCH v2 2/8] tests: Add ovs-barrier unit test

2021-05-20 Thread Gaetan Rivet
No unit test exist currently for the ovs-barrier type. It is however crucial as a building block and should be verified to work as expected. Create a simple test verifying the basic function of ovs-barrier. Integrate the test as part of the test suite. Signed-off-by: Gaetan Rivet --- tests

[ovs-dev] [PATCH v2 5/8] ovs-thread: Fix barrier use-after-free

2021-05-20 Thread Gaetan Rivet
immediately after blocking on it. Fixes: d8043da7182a ("ovs-thread: Implement OVS specific barrier.") Signed-off-by: Gaetan Rivet --- lib/ovs-thread.c | 61 +++- lib/ovs-thread.h | 6 ++--- 2 files changed, 53 insertions(+), 14 deletions(-) di

[ovs-dev] [PATCH v1 7/8] ovs-rcu: Remove unused perthread mutex

2021-04-27 Thread Gaetan Rivet
A mutex is allocated, initialized and destroyed, without being used in the perthread structure. Signed-off-by: Gaetan Rivet --- lib/ovs-rcu.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/lib/ovs-rcu.c b/lib/ovs-rcu.c index cde1e925b..1866bd308 100644 --- a/lib/ovs-rcu.c +++ b/lib/ovs

[ovs-dev] [PATCH v1 6/8] ovs-thread: Quiesce when joining pthreads

2021-04-27 Thread Gaetan Rivet
Joining pthreads makes the caller quiescent. It should register as such, as joined threads may wait on an RCU callback executing before quitting, deadlocking the caller. Signed-off-by: Gaetan Rivet --- lib/ovs-thread.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deletions

[ovs-dev] [PATCH v1 5/8] ovs-thread: Fix barrier use-after-free

2021-04-27 Thread Gaetan Rivet
immediately after blocking on it. Fixes: d8043da7182a ("ovs-thread: Implement OVS specific barrier.") Signed-off-by: Gaetan Rivet --- lib/ovs-thread.c | 61 +++- lib/ovs-thread.h | 6 ++--- 2 files changed, 53 insertions(+), 14 deletions(-) di

[ovs-dev] [PATCH v1 0/8] RCU: Add blocking mode for debugging

2021-04-27 Thread Gaetan Rivet
-free from ASAN. They are however thwarted by the RCU, until the blocking mode is enabled. In that case, they will always abort on the expected error. The full test-suite can be passed with the blocking RCU mode enabled. An entry in the CI matrix is created for it. No error has been observed. Gaetan

[ovs-dev] [PATCH v1 8/8] ovs-rcu: Add blocking RCU mode

2021-04-27 Thread Gaetan Rivet
-start-end # The testsuite can be used as well make check TESTSUITEFLAGS='-k rcu' Signed-off-by: Gaetan Rivet --- .ci/linux-build.sh | 4 ++ .github/workflows/build-and-test.yml | 7 +++ NEWS | 1 + ac

[ovs-dev] [PATCH v1 4/8] tests: Add ASAN use-after-free validation with RCU

2021-04-27 Thread Gaetan Rivet
-off-by: Gaetan Rivet --- tests/automake.mk| 1 + tests/library.at | 33 +++ tests/test-rcu-uaf.c | 98 3 files changed, 132 insertions(+) create mode 100644 tests/test-rcu-uaf.c diff --git a/tests/automake.mk b/tests/automake.mk

[ovs-dev] [PATCH v1 3/8] tests: Add RCU postpone test

2021-04-27 Thread Gaetan Rivet
Add a simple postponing test verifying RCU callbacks have executed and RCU exits in order. Add as part of library unit-tests. Signed-off-by: Gaetan Rivet --- tests/library.at | 8 ++- tests/test-rcu.c | 59 2 files changed, 66 insertions

[ovs-dev] [PATCH v1 1/8] configure: add --enable-asan option

2021-04-27 Thread Gaetan Rivet
Add a configure option to enable ASAN in a simple way. Adding an AC variable to allow checking for support in the testsuite. Signed-off-by: Gaetan Rivet --- .ci/linux-build.sh | 4 ++-- NEWS | 1 + acinclude.m4 | 16 configure.ac | 1 + tests

[ovs-dev] [PATCH v1 2/8] tests: Add ovs-barrier unit test

2021-04-27 Thread Gaetan Rivet
No unit test exist currently for the ovs-barrier type. It is however crucial as a building block and should be verified to work as expected. Create a simple test verifying the basic function of ovs-barrier. Integrate the test as part of the test suite. Signed-off-by: Gaetan Rivet --- tests

[ovs-dev] [PATCH v3 27/28] dpif-netdev: Use one or more offload threads

2021-04-25 Thread Gaetan Rivet
multiplied by the number of requested threads and used separately. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 304 +- lib/netdev-offload-dpdk.c | 7 +- 2 files changed, 204 insertions

  1   2   3   >