[dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0
> -Original Message- > From: Hiroshi Shimamoto [mailto:h-shimamoto at ct.jp.nec.com] > Sent: Friday, September 12, 2014 12:48 AM > To: Richardson, Bruce; dev at dpdk.org > Subject: RE: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to > lcore 0 > > Hi Bruce, > > > Subject: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to > > lcore 0 > > > > This patchset sets things up so that we can affinitize the interrupt, > > vfio management, and hpet timer management threads to lcore 0, so that > > they never interfere with data plane threads. > > I don't think it works well always. > The management threads can be floating all cpus on demand, because those > threads are created before the master thread affinity is set. The kernel > scheduler will take care of it. And we should isolate cpus which data plane > threads are pinned to, so the management threads cannot run on those isolated > cpus data plane thread run. Yes, this was my understanding as well of how things would work - we just had a report back of where the management threads were using the same core as the data-plane despite the core being isolated. I'll double-check to see if this really is happening and resubmit a new patch with a configurable core assignment if it is definitely needed. /Bruce > In some cases, the user may run data plane thread on lcore 0, but with > this patchset the data plane pinned to lcore 0 always run with the > management threads. That doesn't seem good. > > I think this functionality should be conditional. > How about to add a parameter to specify the mask for the management threads > instead of statically assignment to lcore 0? > > thanks, > Hiroshi > > > > > Bruce Richardson (3): > > eal: add core id param to eal_thread_set_affinity > > eal: increase scope of eal_thread_set_affinity > > eal: affinitize low-priority threads to lcore 0 > > > > lib/librte_eal/bsdapp/eal/eal_thread.c | 12 ++-- > > lib/librte_eal/common/include/eal_private.h| 10 ++ > > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 + > > lib/librte_eal/linuxapp/eal/eal_pci_vfio_mp_sync.c | 6 ++ > > lib/librte_eal/linuxapp/eal/eal_thread.c | 12 ++-- > > lib/librte_eal/linuxapp/eal/eal_timer.c| 5 + > > 6 files changed, 38 insertions(+), 12 deletions(-) > > > > -- > > 1.9.3
[dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0
Hi Bruce, > Subject: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0 > > This patchset sets things up so that we can affinitize the interrupt, > vfio management, and hpet timer management threads to lcore 0, so that > they never interfere with data plane threads. I don't think it works well always. The management threads can be floating all cpus on demand, because those threads are created before the master thread affinity is set. The kernel scheduler will take care of it. And we should isolate cpus which data plane threads are pinned to, so the management threads cannot run on those isolated cpus data plane thread run. In some cases, the user may run data plane thread on lcore 0, but with this patchset the data plane pinned to lcore 0 always run with the management threads. That doesn't seem good. I think this functionality should be conditional. How about to add a parameter to specify the mask for the management threads instead of statically assignment to lcore 0? thanks, Hiroshi > > Bruce Richardson (3): > eal: add core id param to eal_thread_set_affinity > eal: increase scope of eal_thread_set_affinity > eal: affinitize low-priority threads to lcore 0 > > lib/librte_eal/bsdapp/eal/eal_thread.c | 12 ++-- > lib/librte_eal/common/include/eal_private.h| 10 ++ > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 + > lib/librte_eal/linuxapp/eal/eal_pci_vfio_mp_sync.c | 6 ++ > lib/librte_eal/linuxapp/eal/eal_thread.c | 12 ++-- > lib/librte_eal/linuxapp/eal/eal_timer.c| 5 + > 6 files changed, 38 insertions(+), 12 deletions(-) > > -- > 1.9.3
[dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0
On Thu, 11 Sep 2014 23:47:52 + Hiroshi Shimamoto wrote: > Hi Bruce, > > > Subject: [dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to > > lcore 0 > > > > This patchset sets things up so that we can affinitize the interrupt, > > vfio management, and hpet timer management threads to lcore 0, so that > > they never interfere with data plane threads. > > I don't think it works well always. > The management threads can be floating all cpus on demand, because those > threads are created before the master thread affinity is set. The kernel > scheduler will take care of it. And we should isolate cpus which data plane > threads are pinned to, so the management threads cannot run on those isolated > cpus data plane thread run. > In some cases, the user may run data plane thread on lcore 0, but with > this patchset the data plane pinned to lcore 0 always run with the > management threads. That doesn't seem good. > > I think this functionality should be conditional. > How about to add a parameter to specify the mask for the management threads > instead of statically assignment to lcore 0? > > thanks, > Hiroshi > > > > > Bruce Richardson (3): > > eal: add core id param to eal_thread_set_affinity > > eal: increase scope of eal_thread_set_affinity > > eal: affinitize low-priority threads to lcore 0 > > > > lib/librte_eal/bsdapp/eal/eal_thread.c | 12 ++-- > > lib/librte_eal/common/include/eal_private.h| 10 ++ > > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 + > > lib/librte_eal/linuxapp/eal/eal_pci_vfio_mp_sync.c | 6 ++ > > lib/librte_eal/linuxapp/eal/eal_thread.c | 12 ++-- > > lib/librte_eal/linuxapp/eal/eal_timer.c| 5 + > > 6 files changed, 38 insertions(+), 12 deletions(-) > > > > -- > > 1.9.3 > We just set the non-master DPDK worker threads to be real time by using pthread_set_scheduler. This works very well and makes it clear to Linux scheduler what the intention is. This also improves wakeup behavior from sleeps.
[dpdk-dev] [PATCH 0/3] eal affinitize low priority threads to lcore 0
This patchset sets things up so that we can affinitize the interrupt, vfio management, and hpet timer management threads to lcore 0, so that they never interfere with data plane threads. Bruce Richardson (3): eal: add core id param to eal_thread_set_affinity eal: increase scope of eal_thread_set_affinity eal: affinitize low-priority threads to lcore 0 lib/librte_eal/bsdapp/eal/eal_thread.c | 12 ++-- lib/librte_eal/common/include/eal_private.h| 10 ++ lib/librte_eal/linuxapp/eal/eal_interrupts.c | 5 + lib/librte_eal/linuxapp/eal/eal_pci_vfio_mp_sync.c | 6 ++ lib/librte_eal/linuxapp/eal/eal_thread.c | 12 ++-- lib/librte_eal/linuxapp/eal/eal_timer.c| 5 + 6 files changed, 38 insertions(+), 12 deletions(-) -- 1.9.3
