Bhanuprakash Bodireddy <bhanuprakash.bodire...@intel.com> writes:
> Increase the DPDK pmd thread scheduling priority by lowering the nice
> value. This will advise the kernel scheduler to prioritize pmd thread
> over other processes.
>
> Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodire...@intel.com>
> ---
Sorry for jumping into this so late. Is there a measured benefit to
this patch? Do you have a test case to reproduce the effect you're
seeing? Might it be better to write up documentation for the user
describing chrt/nice/renice utilities?
> v7->v8:
> * Rebase
> * Update the documentation file @Documentation/intro/install/dpdk-advanced.rst
>
> v6->v7:
> * Remove realtime scheduling policy logic.
> * Increase pmd thread scheduling priority by lowering nice value to -20.
> * Update doc accordingly.
>
> v5->v6:
> * Prohibit spawning pmd thread on the lowest core in dpdk-lcore-mask if
> lcore-mask and pmd-mask affinity are identical.
> * Updated Note section in INSTALL.DPDK-ADVANCED doc.
> * Tested below cases to verify system stability with pmd priority patch
>
> v4->v5:
> * Reword Note section in DPDK-ADVANCED.md
>
> v3->v4:
> * Document update
> * Use ovs_strerror for reporting errors in lib-numa.c
>
> v2->v3:
> * Move set_priority() function to lib/ovs-numa.c
> * Apply realtime scheduling policy and priority to pmd thread only if
> pmd-cpu-mask is passed.
> * Update INSTALL.DPDK-ADVANCED.
>
> v1->v2:
> * Removed #ifdef and introduced dummy function "pmd_thread_setpriority"
> in netdev-dpdk.h
> * Rebase
>
> Documentation/intro/install/dpdk-advanced.rst | 8 +++++++-
> lib/dpif-netdev.c | 4 ++++
> lib/ovs-numa.c | 19 +++++++++++++++++++
> lib/ovs-numa.h | 1 +
> 4 files changed, 31 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/intro/install/dpdk-advanced.rst
> b/Documentation/intro/install/dpdk-advanced.rst
> index 44d1cd7..67815ac 100644
> --- a/Documentation/intro/install/dpdk-advanced.rst
> +++ b/Documentation/intro/install/dpdk-advanced.rst
> @@ -238,7 +238,8 @@ affinitized accordingly.
> to be affinitized to isolated cores for optimum performance.
>
> By setting a bit in the mask, a pmd thread is created and pinned to the
> - corresponding CPU core. e.g. to run a pmd thread on core 2::
> + corresponding CPU core and the nice value set to '-20'.
> + e.g. to run a pmd thread on core 2::
>
> $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x4
>
> @@ -278,6 +279,11 @@ improvements as there will be more total CPU occupancy
> available::
>
> NIC port0 <-> OVS <-> VM <-> OVS <-> NIC port 1
>
> + .. note::
> + It is recommended that the OVS control thread and pmd thread shouldn't be
> + pinned to the same core i.e 'dpdk-lcore-mask' and 'pmd-cpu-mask' cpu mask
> + settings should be non-overlapping.
> +
> DPDK Physical Port Rx Queues
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> index 0b73056..f58a855 100644
> --- a/lib/dpif-netdev.c
> +++ b/lib/dpif-netdev.c
> @@ -3120,6 +3120,10 @@ pmd_thread_main(void *f_)
> ovs_numa_thread_setaffinity_core(pmd->core_id);
> dpdk_set_lcore_id(pmd->core_id);
> poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list);
> +
> + /* Set pmd thread's nice value to -20 */
> +#define MIN_NICE -20
> + ovs_numa_thread_setpriority(MIN_NICE);
> reload:
> emc_cache_init(&pmd->flow_cache);
>
> diff --git a/lib/ovs-numa.c b/lib/ovs-numa.c
> index e1e7068..d52c517 100644
> --- a/lib/ovs-numa.c
> +++ b/lib/ovs-numa.c
> @@ -23,6 +23,7 @@
> #include <dirent.h>
> #include <stddef.h>
> #include <string.h>
> +#include <sys/resource.h>
> #include <sys/types.h>
> #include <unistd.h>
> #endif /* __linux__ */
> @@ -620,3 +621,21 @@ int ovs_numa_thread_setaffinity_core(unsigned core_id
> OVS_UNUSED)
> return EOPNOTSUPP;
> #endif /* __linux__ */
> }
> +
> +void
> +ovs_numa_thread_setpriority(int nice OVS_UNUSED)
> +{
> + if (dummy_numa) {
> + return;
> + }
> +
> +#ifndef _WIN32
> + int err;
> + err = setpriority(PRIO_PROCESS, 0, nice);
> + if (err) {
> + VLOG_ERR("Thread priority error %s",ovs_strerror(err));
> + }
> +#else
> + return EOPNOTSUPP;
> +#endif
> +}
> diff --git a/lib/ovs-numa.h b/lib/ovs-numa.h
> index be836b2..3ce13c7 100644
> --- a/lib/ovs-numa.h
> +++ b/lib/ovs-numa.h
> @@ -56,6 +56,7 @@ void ovs_numa_unpin_core(unsigned core_id);
> struct ovs_numa_dump *ovs_numa_dump_cores_on_numa(int numa_id);
> void ovs_numa_dump_destroy(struct ovs_numa_dump *);
> int ovs_numa_thread_setaffinity_core(unsigned core_id);
> +void ovs_numa_thread_setpriority(int nice);
>
> #define FOR_EACH_CORE_ON_NUMA(ITER, DUMP) \
> LIST_FOR_EACH((ITER), list_node, &(DUMP)->dump)
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
