Sure, I think it is possible as only few batches needed for typical workload. For dynamically mapping extended memory, I think this function is still needed as new segment will be allocated from system memory when new stream established. This action is happened in work thread.
> -----Original Message----- > From: Damjan Marion <[email protected]> > Sent: Wednesday, November 30, 2022 9:23 PM > To: Liu, Yong <[email protected]> > Cc: [email protected] > Subject: Re: Gerrit review for memif DMA acceleration > > > Thanks, > > dynamically allocating physical memory from worker thread is not something > what we do today and i don’t think it is right way to do. > Even for buffer pools we don’t do that. Can you simply pre-allocate > reasonable amount of physical memory on startup instead? > > — > Damjan > > > > On 30.11.2022., at 10:20, Liu, Yong <[email protected]> wrote: > > > > Hi Damjan, > > VFIO map function now can be called from work thread in some cases. Like > allocate physical memory for DMA batch and then do mapping for if no > attached device, or a new session segment attached to VPP process. So I use > event-logger for vfio logging. > > These numbers are collected from my sample server, we are modifying > CSTI case for DMA usage. Will see more official number later. > > > > 1C memif l2patch No-DMA > > 64 128 256 512 1024 > > 1518 > > 8.00Mpps 6.49Mpps 4.69Mpps 3.23Mpps 2.37Mpps 1.96Mpps > > 4.09Gbps 6.65Gbps 9.62Gbps 13.24Gbps 19.43Gbps 23.86Gbps > > > > 1C memif l2patch DMA > > 64 128 256 512 1024 > > 1518 > > 8.65Mpps 8.60Mpps 8.54Mpps 8.22Mpps 8.36Mpps 7.61Mpps > > 4.43Gbps 8.81Gbps 8.54Mpps 33.67Gbps 68.51Gbps 92.39Gbps > > > > Regards, > > Marvin > > > >> -----Original Message----- > >> From: Damjan Marion <[email protected]> > >> Sent: Tuesday, November 29, 2022 10:45 PM > >> To: Liu, Yong <[email protected]> > >> Cc: [email protected] > >> Subject: Re: Gerrit review for memif DMA acceleration > >> > >> > >> Hi Marvin, > >> > >> for a start can you use standard vlib logging instead of elog, as all those > >> logging stiuff are not perf critical. > >> > >> Also, can you share some perf comparison between standard CPU path > and > >> DSA accelerated memif? > >> > >> Thanks, > >> > >> Damjan > >> > >> > >>> On 29.11.2022., at 09:05, Liu, Yong <[email protected]> wrote: > >>> > >>> Hi Damjan and community, > >>> For extend the usage of latest introduced DMA infrastructure, I uploaded > >> several patches for review. Before your review, let me briefly introduce > >> these patches. > >>> In review 37572, add a new vfio mapped function for extended memory. > >> This kind of memory may come from another process like memif regions > or > >> dynamic allocated like hoststack shared segments. > >>> In review 37573, support vfio based DSA device for the scenario that > need > >> fully control the resource of DSA. Compared to assign work queues from > >> single idxd instance into multiple processes, this way can guarantee > resource. > >>> In review 37574, support CBDMA device which only support pci device > >> model. The usage of CBDMA in hoststack and memif is depend on 37572. > >>> In review 37572, add new datapath function in memif input and tx node. > >> These functions followed async model and will be chosen if option > “use_dma” > >> added when creating memif interface. > >>> Gerrit link: > >>> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgerr > it.fd.io%2Fr%2Fc%2Fvpp%2F%2B%2F37572&data=05%7C01%7C%7C93c > 7d85e4b704701d79e08dad2b422f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa > %7C1%7C0%7C638053968361772272%7CUnknown%7CTWFpbGZsb3d8eyJWI > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3 > 000%7C%7C%7C&sdata=92BzqQfssl2f5%2BufcV1kLyBgGC%2BOGIf5uAG > Iz27u8jg%3D&reserved=0 > >>> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgerr > it.fd.io%2Fr%2Fc%2Fvpp%2F%2B%2F37573&data=05%7C01%7C%7C93c > 7d85e4b704701d79e08dad2b422f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa > %7C1%7C0%7C638053968361772272%7CUnknown%7CTWFpbGZsb3d8eyJWI > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3 > 000%7C%7C%7C&sdata=PRz65kWJOm6APSaNph0GSrGqxCz6S6iG9LkliD > pJf5E%3D&reserved=0 > >>> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgerr > it.fd.io%2Fr%2Fc%2Fvpp%2F%2B%2F37574&data=05%7C01%7C%7C93c > 7d85e4b704701d79e08dad2b422f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa > %7C1%7C0%7C638053968361772272%7CUnknown%7CTWFpbGZsb3d8eyJWI > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3 > 000%7C%7C%7C&sdata=uVO7nnt5%2F%2BmSnUpjjol7H24L%2BJ1%2BL > o9PaQ3esHca10w%3D&reserved=0 > >>> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgerr > it.fd.io%2Fr%2Fc%2Fvpp%2F%2B%2F37731&data=05%7C01%7C%7C93c > 7d85e4b704701d79e08dad2b422f7%7C84df9e7fe9f640afb435aaaaaaaaaaaa > %7C1%7C0%7C638053968361772272%7CUnknown%7CTWFpbGZsb3d8eyJWI > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3 > 000%7C%7C%7C&sdata=j1IfYk50LW17bgTcDWTZB5L97vhMB9AE6%2FBt > LkTbNs4%3D&reserved=0 > >>> Best Regards, > >>> Marvin > >> > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22262): https://lists.fd.io/g/vpp-dev/message/22262 Mute This Topic: https://lists.fd.io/mt/95330357/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
