Simply reordering the launch of different threads brings back a lot of the lost 
performance, this is a clear evidence that some CPU threads are more 
predisposed to context switches than the others.

This is a thread scheduling issue at the CPU level as we have expected. In a 
previous exchange someone has suggested that utilizing rte_thread_set_priority 
to RTE_THREAD_PRIORITY_REALTIME_CRITICAL is not a good idea 

we should be able to prioritize some threads over the other threads ... since 
we are utilizing rte_eal_remote_launch, one would think that such a 
functonality should be a part of the library ...

any ideas folks?

regards






On Tuesday, September 24, 2024 at 01:47:05 PM PDT, amit sehas <cu...@yahoo.com> 
wrote: 





Thanks for the suggestions, so this is a database server which is doing lots of 
stuff, not every thread is heavily involved in dpdk packet processing. As a 
result the guidelines for attaining the most dpdk performance are applicable to 
only a few threads.

In this particular issue we are specificially looking at CPU scheduling of 
threads that are primarily heavily processing database queries. These threads, 
from our measurements, are not being uniformly scheduled on the CPU ...

This is our primary concern, since we utilized rte_eal_remote_launch to spawn 
the threads, we are wondering if there are any options in this API that will 
allow us to more uniformly allocate the CPU to threads that are critical...

regards


On Tuesday, September 24, 2024 at 09:38:16 AM PDT, Stephen Hemminger 
<step...@networkplumber.org> wrote: 





On Tue, 24 Sep 2024 14:40:49 +0000 (UTC)

amit sehas <cu...@yahoo.com> wrote:

> Thanks for your response, and thanks for your input on the set_priority, 
> 
> The best guess we have at this point is that this is not a dpdk performance 
> issue. This is an issue with some threads running into more context switches 
> than the others and hence not getting the same slice of the CPU. We are 
> certain that this is not a dpdk performance issue, the code
> is uniformly slow in one thread versus the other and the threads are doing a 
> very large amount of work including accessing databases. The threads in 
> question are not really doing packet processing as much as other work.
> 
> So this is certainly not a dpdk performance issue. This is an issue of kernel 
> threads not being scheduled properly or in the worse case the cores running 
> on different frequency (which is quite unlikely no the AWS Xeons we are 
> running this on).
> 
> If you are asking for the dpdk config files to check for dpdk related 
> performance issue then we are quite certain the issue is not with dpdk 
> performance ...

> On Mon, Sep 23, 2024 at 10:06 PM amit sehas <cu...@yahoo.com> wrote:
> > Thanks for your response, i am not sure i understand your question ... we 
> > have our product that utilizes dpdk ... the commands are just our server 
> > commands and parameters ... and the lscpu is the hyperthreaded 8 thread 
> > Xeon instance in AWS ...



The rules of getting performance in DPDK:
  - use DPDK threads (pinned) for datapath
  - use isolated CPU's for those DPDK threads
  - do not do any system calls
  - avoid floating point

You can use tracing tools like strace or BPF to see what the thread is doing.

Reply via email to