Hi Cunming, The timers could be used by any application/library started as a standard pthread. Each pthread needs to have assigned some identifier same way as you are doing it for mempools (the rte_linear_thread_id and rte_lcore_id are good examples)
I made series of patches extending the rte timers API to use with such kind of identifier keeping existing API working also. I will send it soon. Mirek > -----Original Message----- > From: Liang, Cunming > Sent: Friday, December 12, 2014 6:45 AM > To: Walukiewicz, Miroslaw; dev at dpdk.org > Subject: RE: [dpdk-dev] [RFC PATCH 0/7] support multi-phtread per lcore > > Thanks Mirek. That's a good point which wasn't mentioned in cover letter. > For 'rte_timer', I only expect it be used within the 'legacy per-lcore' > pthread. > I'm appreciate if you can give me some cases which can't use it to fit. > In case have to use 'rte_timer' in multi-pthread, there are some > prerequisites and limitations. > 1. Make sure thread local variable 'lcore_id' is set correctly (e.g. do > pthread > init by rte_pthread_prepare) > 2. As 'rte_timer' is not preemptable, when using rte_timer_manager/reset in > multi-pthread, make sure they're not on the same core. > > -Cunming > > > -----Original Message----- > > From: Walukiewicz, Miroslaw > > Sent: Thursday, December 11, 2014 5:57 PM > > To: Liang, Cunming; dev at dpdk.org > > Subject: RE: [dpdk-dev] [RFC PATCH 0/7] support multi-phtread per lcore > > > > Thank you Cunming for explanation. > > > > What about DPDK timers? They also depend on rte_lcore_id() to avoid > spinlocks. > > > > Mirek > > > > > -----Original Message----- > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Cunming Liang > > > Sent: Thursday, December 11, 2014 3:05 AM > > > To: dev at dpdk.org > > > Subject: [dpdk-dev] [RFC PATCH 0/7] support multi-phtread per lcore > > > > > > > > > Scope & Usage Scenario > > > ======================== > > > > > > DPDK usually pin pthread per core to avoid task switch overhead. It gains > > > performance a lot, but it's not efficient in all cases. In some cases, it > > > may > > > too expensive to use the whole core for a lightweight workload. It's a > > > reasonable demand to have multiple threads per core and each threads > > > share CPU > > > in an assigned weight. > > > > > > In fact, nothing avoid user to create normal pthread and using cgroup to > > > control the CPU share. One of the purpose for the patchset is to clean the > > > gaps of using more DPDK libraries in the normal pthread. In addition, it > > > demonstrates performance gain by proactive 'yield' when doing idle loop > > > in packet IO. It also provides several 'rte_pthread_*' APIs to easy life. > > > > > > > > > Changes to DPDK libraries > > > ========================== > > > > > > Some of DPDK libraries must run in DPDK environment. > > > > > > # rte_mempool > > > > > > In rte_mempool doc, it mentions a thread not created by EAL must not > use > > > mempools. The root cause is it uses a per-lcore cache inside mempool. > > > And 'rte_lcore_id()' will not return a correct value. > > > > > > The patchset changes this a little. The index of mempool cache won't be a > > > lcore_id. Instead of it, using a linear number generated by the allocator. > > > For those legacy EAL per-lcore thread, it apply for an unique linear id > > > during creation. For those normal pthread expecting to use > rte_mempool, it > > > requires to apply for a linear id explicitly. Now the mempool cache looks > like > > > a per-thread base. The linear ID actually identify for the linear thread > > > id. > > > > > > However, there's another problem. The rte_mempool is not > preemptable. > > > The > > > problem comes from rte_ring, so talk together in next section. > > > > > > # rte_ring > > > > > > rte_ring supports multi-producer enqueue and multi-consumer > dequeue. > > > But it's > > > not preemptable. There's conversation talking about this before. > > > http://dpdk.org/ml/archives/dev/2013-November/000714.html > > > > > > Let's say there's two pthreads running on the same core doing enqueue > on > > > the > > > same rte_ring. If the 1st pthread is preempted by the 2nd pthread while > it > > > has > > > already modified the prod.head, the 2nd pthread will spin until the 1st > one > > > scheduled agian. It causes time wasting. In addition, if the 2nd pthread > has > > > absolutely higer priority, it's more terrible. > > > > > > But it doesn't means we can't use. Just need to narrow down the > situation > > > when > > > it's used by multi-pthread on the same core. > > > - It CAN be used for any single-producer or single-consumer situation. > > > - It MAY be used by multi-producer/consumer pthread whose scheduling > > > policy > > > are all SCHED_OTHER(cfs). User SHOULD aware of the performance > penalty > > > befor > > > using it. > > > - It MUST not be used by multi-producer/consumer pthread, while some > of > > > their > > > scheduling policies is SCHED_FIFO or SCHED_RR. > > > > > > > > > Performance > > > ============== > > > > > > It loses performance by introducing task switching. On packet IO > perspective, > > > we can gain some back by improving IO effective rate. When the pthread > do > > > idle > > > loop on an empty rx queue, it should proactively yield. We can also slow > > > down > > > rx for a bit while to take more advantage of the bulk receiving in the > > > next > > > loop. In practice, increase the rx ring size also helps to improve the > overrall > > > throughput. > > > > > > > > > Cgroup Control > > > ================ > > > > > > Here's a simple example, there's four pthread doing packet IO on the > same > > > core. > > > We expect the CPU share rate is 1:1:2:4. > > > > mkdir /sys/fs/cgroup/cpu/dpdk > > > > mkdir /sys/fs/cgroup/cpu/dpdk/thread0 > > > > mkdir /sys/fs/cgroup/cpu/dpdk/thread1 > > > > mkdir /sys/fs/cgroup/cpu/dpdk/thread2 > > > > mkdir /sys/fs/cgroup/cpu/dpdk/thread3 > > > > cd /sys/fs/cgroup/cpu/dpdk > > > > echo 256 > thread0/cpu.shares > > > > echo 256 > thread1/cpu.shares > > > > echo 512 > thread2/cpu.shares > > > > echo 1024 > thread3/cpu.shares > > > > > > > > > -END- > > > > > > Any comments are welcome. > > > > > > Thanks > > > > > > *** BLURB HERE *** > > > > > > Cunming Liang (7): > > > eal: add linear thread id as pthread-local variable > > > mempool: use linear-tid as mempool cache index > > > ring: use linear-tid as ring debug stats index > > > eal: add simple API for multi-pthread > > > testpmd: support multi-pthread mode > > > sample: add new sample for multi-pthread > > > eal: macro for cpuset w/ or w/o CPU_ALLOC > > > > > > app/test-pmd/cmdline.c | 41 +++++ > > > app/test-pmd/testpmd.c | 84 ++++++++- > > > app/test-pmd/testpmd.h | 1 + > > > config/common_linuxapp | 1 + > > > examples/multi-pthread/Makefile | 57 ++++++ > > > examples/multi-pthread/main.c | 232 > > ++++++++++++++++++++++++ > > > examples/multi-pthread/main.h | 46 +++++ > > > lib/librte_eal/common/include/rte_eal.h | 15 ++ > > > lib/librte_eal/common/include/rte_lcore.h | 12 ++ > > > lib/librte_eal/linuxapp/eal/eal_thread.c | 282 > > > +++++++++++++++++++++++++++--- > > > lib/librte_mempool/rte_mempool.h | 22 +-- > > > lib/librte_ring/rte_ring.h | 6 +- > > > 12 files changed, 755 insertions(+), 44 deletions(-) > > > create mode 100644 examples/multi-pthread/Makefile > > > create mode 100644 examples/multi-pthread/main.c > > > create mode 100644 examples/multi-pthread/main.h > > > > > > -- > > > 1.8.1.4