[dpdk-dev] Vmxnet3-pmd not receiving packets after random time under high load
I dug deeper into the problem vmxnet-pmd not capturing packets. Whenever pmd
stops capturing, it does not respect the number of rxd. For example, I set rxd
to 512. The pkt mbuf mempool I allocate is 4482, which is much bigger than rxd.
I would expect ~512 mbufs to be removed from mempool at any moment in time.
When not able to capture new packets is imminent, pmd would use much more than
512 to the point of exhausting all the mbufs in mempool. After this point, pmd
stops capturing packet and does not recover; it never enters the while loop
below.
vmxnet3_rxtx.c
457 while (rcd->gen == rxq->comp_ring.gen) {
This issue occurs quite randomly, which leads me to believe there is some race
condition. It occurs when packets are arriving faster than the application can
process.
I understand that vmxnet3 pmd preallocates rxq descriptors during
initialization. Yet, even when it?s receiving packets correctly, it uses a
little more than rxq descriptors. I find the number of mbufs in mempool very
interesting.
Before vmxnet3 init
4482
After vmxnet3 init
3970
When receiving packets properly
ranges between 3920 - 3970
Prior to stop receiving packets
ranges between 0 and 3970
After the last packet is received
4481
I appreciate any comment/help on this. Thanks.
Dan
On Mar 11, 2014, at 12:55 PM, Daniel Kan wrote:
> I?m unable to get RSS to work properly with vmxnet3-pmd. The first issue is
> that the number of rxqs must be power of 2. Otherwise, rte_eth_dev_start()
> fails due to inability to activate vmxnet3 NIC. This is not too big of a
> deal, but physical NICs don?t have this requirement.
>
> The second issue is that RSS is just not working at all for me. The rxmode is
> set to ETH_MQ_RX_RSS and rss_hf = ETH_RSS_IPV4_TCP | ETH_RSS_IPV4_UDP |
> ETH_RSS_IPV4 | ETH_RSS_IPV6. The same configuration works for a real NIC.
> When I checked mb->pkt.hash, the value is all zeroed out.
>
> Even if I disabled RSS, I found the performance of vmxnet3-pmd to be quite
> poor, peaking out at 600k pps with 64 byte packet, while libpcap can do 650k
> pps.
>
> Lastly, there is a stability issue. On a number of occasions, vmxnet3-pmd
> stops receiving packets after some random time and several million packets.
>
> I?m not sure if anyone else is having as much issue as I?m, I will give
> vmxnet3-usermap a try.
>
> Finally, does either vmxnet3-usermap or vmxnet3-pmd work well for
> non-Intel-based underlying physical NIC?
>
> Thanks.
>
> Dan
>
>
[dpdk-dev] [PATCH] vmxnet3: fixed segfault when initializing vmxnet3 pmd on linux platform
The vmxnet3 PCI hardware resources were never memory mapped when
RE_EAL_UNBIND_PORTS is not defined.
Specifically, pci_dev->mem_resource is not mapped. The fix is to always set
drv_flags with RTE_PCI_DRV_NEED_IGU for vmxnet3. This ensures
pci_uio_map_resource() is called.
Signed-off-by: Daniel Kan
---
lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c |2 --
1 file changed, 2 deletions(-)
diff --git a/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
b/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
index 6757aa2..8259cfe 100644
--- a/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
+++ b/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
@@ -267,9 +267,7 @@ static struct eth_driver rte_vmxnet3_pmd = {
{
.name = "rte_vmxnet3_pmd",
.id_table = pci_id_vmxnet3_map,
-#ifdef RTE_EAL_UNBIND_PORTS
.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
-#endif
},
.eth_dev_init = eth_vmxnet3_dev_init,
.dev_private_size = sizeof(struct vmxnet3_adapter),
--
1.7.9.5
[dpdk-dev] RSS, performance, and stability issues vmxnet3-pmd
I?m unable to get RSS to work properly with vmxnet3-pmd. The first issue is that the number of rxqs must be power of 2. Otherwise, rte_eth_dev_start() fails due to inability to activate vmxnet3 NIC. This is not too big of a deal, but physical NICs don?t have this requirement. The second issue is that RSS is just not working at all for me. The rxmode is set to ETH_MQ_RX_RSS and rss_hf = ETH_RSS_IPV4_TCP | ETH_RSS_IPV4_UDP | ETH_RSS_IPV4 | ETH_RSS_IPV6. The same configuration works for a real NIC. When I checked mb->pkt.hash, the value is all zeroed out. Even if I disabled RSS, I found the performance of vmxnet3-pmd to be quite poor, peaking out at 600k pps with 64 byte packet, while libpcap can do 650k pps. Lastly, there is a stability issue. On a number of occasions, vmxnet3-pmd stops receiving packets after some random time and several million packets. I?m not sure if anyone else is having as much issue as I?m, I will give vmxnet3-usermap a try. Finally, does either vmxnet3-usermap or vmxnet3-pmd work well for non-Intel-based underlying physical NIC? Thanks. Dan
[dpdk-dev] On vmxnet-pmd crash in DPDK 1.6.0r1
virtio-pmd has the same pattern. I wonder if vmxnet3-pmd just blindly copied
the same paradigm.
lib/librte_pmd_virtio/virtio_ethdev.c
473 static struct eth_driver rte_virtio_pmd = {
474 {
475 .name = "rte_virtio_pmd",
476 .id_table = pci_id_virtio_map,
477 #ifdef RTE_EAL_UNBIND_PORTS
478 .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
479 #endif
480 },
481 .eth_dev_init = eth_virtio_dev_init,
482 .dev_private_size = sizeof(struct virtio_adapter),
483 };
On Mar 10, 2014, at 11:20 PM, Daniel Kan wrote:
> Upon further trace, I know what caused it. The uio hardware resources were
> never memory mapped when RTE_EAL_UNBIND_PORTS is not enabled. Specifically,
> pci_dev->mem_resource[] is not mapped. This explains why setting
> CONFIG_RTE_EAL_UNBIND_PORTS=y fixes the problem.
>
> lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
> 266 static struct eth_driver rte_vmxnet3_pmd = {
> 267 {
> 268 .name = "rte_vmxnet3_pmd",
> 269 .id_table = pci_id_vmxnet3_map,
> 270 #ifdef RTE_EAL_UNBIND_PORTS
> 271 .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
> 272 #endif
> 273 },
> 274 .eth_dev_init = eth_vmxnet3_dev_init,
> 275 .dev_private_size = sizeof(struct vmxnet3_adapter),
> 276 };
>
> Note drv_flags will be 0.
>
> lib/librte_eal/linuxapp/eal/eal_pci.c
> 1039 #ifdef RTE_EAL_UNBIND_PORTS
> 1040 if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO) {
> 1041 /* unbind driver and load uio resources for Intel NICs */
> 1042 if (pci_switch_module(dr, dev, 1, IGB_UIO_NAME) < 0)
> ...
> 1050 #else
> 1051 if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO)
> 1052 /* just map resources for Intel NICs */
> 1053 if (pci_uio_map_resource(dev) < 0)
> 1054 return -1;
> 1055 #endif
>
> If RTE_EAL_UNBIND_PORTS is defined, pci_switch_module will call
> pci_uio_map_resource.
>
> I then looked at the bsdapp?s version, it simply has:
> lib/librte_eal/bsdapp/eal/eal_pci.c
> 479 /* just map the NIC resources */
> 480 if (pci_uio_map_resource(dev) < 0)
> 481 return -1;
>
> I don?t know the history behind why .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO
> only for RTE_EAL_UNBIND_PORTS. Can we just ensure pci_uio_map_resource is
> called in linux just like bsd version? One way is to ensure drv_flags is
> always set to RTE_PCI_DRV_NEED_IGB_UIO, but I don?t know if this fix will
> break other parts.
>
> +++ vmxnet3_ethdev.c 2014-03-10 23:18:02.087742434 -0700
> @@ -267,9 +267,7 @@
> {
> .name = "rte_vmxnet3_pmd",
> .id_table = pci_id_vmxnet3_map,
> -#ifdef RTE_EAL_UNBIND_PORTS
> .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
> -#endif
> },
> .eth_dev_init = eth_vmxnet3_dev_init,
> .dev_private_size = sizeof(struct vmxnet3_adapter),
>
> Thanks.
>
> Dan
>
> On Mar 10, 2014, at 10:33 PM, Daniel Kan wrote:
>
>> I also got the segfault. Setting CONFIG_RTE_EAL_UNBIND_PORTS=y is not ideal
>> because it would also unbind my VMNET3 management interface.
>>
>> Does anyone know why the crash is happening? According to the stacktrace,
>> hw_addrX is 0x0 during eth_vmxnet3_dev_init.
>>
>> I?m running on ubuntu 12.04LTS on ESXi 5.5 with Intel I350 NIC as the
>> physical adapter for VMNET3.
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x004fe0eb in vmxnet3_read_addr (addr=0x0) at
>> /home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h:139
>> (gdb) up
>> #1 0x004fe331 in eth_vmxnet3_dev_init (eth_drv=0x799440
>> , eth_dev=0x7d5280 ) at
>> /home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_v\
>> mxnet3/vmxnet3_ethdev.c:218
>> (gdb) print *hw
>> $12 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968,
>> vendor_id = 5549, subsystem_device_id = 0, subsystem_vendor_id = 0,
>> adapter_stopped = 0, perm_addr = "\000\000\\
>> 000\000\000", num_tx_queues = 1 '\001', num_rx_queues = 1 '\001',
>> bufs_per_pkt = 1 '\001', cur_mtu = 1500, tqd_start = 0x0, rqd_start = 0x0,
>> shared = 0x0, sharedPA = 0, queueDescPA\
>> = 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
>>
>> Thanks in advance.
>>
>> Dan
>>
>> On Mar 10, 2014, at 1:00 AM, Prashant Upadhyaya > aricent.com> wrote:
>>
>>> Hi Srini,
>>>
>>> Thanks, I could also make it work, thanks to your cue !
>>>
>>> Now then, this multi-segment
[dpdk-dev] On vmxnet-pmd crash in DPDK 1.6.0r1
Upon further trace, I know what caused it. The uio hardware resources were
never memory mapped when RTE_EAL_UNBIND_PORTS is not enabled. Specifically,
pci_dev->mem_resource[] is not mapped. This explains why setting
CONFIG_RTE_EAL_UNBIND_PORTS=y fixes the problem.
lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c
266 static struct eth_driver rte_vmxnet3_pmd = {
267 {
268 .name = "rte_vmxnet3_pmd",
269 .id_table = pci_id_vmxnet3_map,
270 #ifdef RTE_EAL_UNBIND_PORTS
271 .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
272 #endif
273 },
274 .eth_dev_init = eth_vmxnet3_dev_init,
275 .dev_private_size = sizeof(struct vmxnet3_adapter),
276 };
Note drv_flags will be 0.
lib/librte_eal/linuxapp/eal/eal_pci.c
1039 #ifdef RTE_EAL_UNBIND_PORTS
1040 if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO) {
1041 /* unbind driver and load uio resources for Intel NICs */
1042 if (pci_switch_module(dr, dev, 1, IGB_UIO_NAME) < 0)
...
1050 #else
1051 if (dr->drv_flags & RTE_PCI_DRV_NEED_IGB_UIO)
1052 /* just map resources for Intel NICs */
1053 if (pci_uio_map_resource(dev) < 0)
1054 return -1;
1055 #endif
If RTE_EAL_UNBIND_PORTS is defined, pci_switch_module will call
pci_uio_map_resource.
I then looked at the bsdapp?s version, it simply has:
lib/librte_eal/bsdapp/eal/eal_pci.c
479 /* just map the NIC resources */
480 if (pci_uio_map_resource(dev) < 0)
481 return -1;
I don?t know the history behind why .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO only
for RTE_EAL_UNBIND_PORTS. Can we just ensure pci_uio_map_resource is called in
linux just like bsd version? One way is to ensure drv_flags is always set to
RTE_PCI_DRV_NEED_IGB_UIO, but I don?t know if this fix will break other parts.
+++ vmxnet3_ethdev.c2014-03-10 23:18:02.087742434 -0700
@@ -267,9 +267,7 @@
{
.name = "rte_vmxnet3_pmd",
.id_table = pci_id_vmxnet3_map,
-#ifdef RTE_EAL_UNBIND_PORTS
.drv_flags = RTE_PCI_DRV_NEED_IGB_UIO,
-#endif
},
.eth_dev_init = eth_vmxnet3_dev_init,
.dev_private_size = sizeof(struct vmxnet3_adapter),
Thanks.
Dan
On Mar 10, 2014, at 10:33 PM, Daniel Kan wrote:
> I also got the segfault. Setting CONFIG_RTE_EAL_UNBIND_PORTS=y is not ideal
> because it would also unbind my VMNET3 management interface.
>
> Does anyone know why the crash is happening? According to the stacktrace,
> hw_addrX is 0x0 during eth_vmxnet3_dev_init.
>
> I?m running on ubuntu 12.04LTS on ESXi 5.5 with Intel I350 NIC as the
> physical adapter for VMNET3.
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x004fe0eb in vmxnet3_read_addr (addr=0x0) at
> /home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h:139
> (gdb) up
> #1 0x004fe331 in eth_vmxnet3_dev_init (eth_drv=0x799440
> , eth_dev=0x7d5280 ) at
> /home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_v\
> mxnet3/vmxnet3_ethdev.c:218
> (gdb) print *hw
> $12 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968,
> vendor_id = 5549, subsystem_device_id = 0, subsystem_vendor_id = 0,
> adapter_stopped = 0, perm_addr = "\000\000\\
> 000\000\000", num_tx_queues = 1 '\001', num_rx_queues = 1 '\001',
> bufs_per_pkt = 1 '\001', cur_mtu = 1500, tqd_start = 0x0, rqd_start = 0x0,
> shared = 0x0, sharedPA = 0, queueDescPA\
> = 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
>
> Thanks in advance.
>
> Dan
>
> On Mar 10, 2014, at 1:00 AM, Prashant Upadhyaya aricent.com> wrote:
>
>> Hi Srini,
>>
>> Thanks, I could also make it work, thanks to your cue !
>>
>> Now then, this multi-segment not being supported in vmxnet3 driver is a big
>> party-pooper for me. Unfortunately in my usecase, I do indeed make heavy use
>> of multisegment buffers for sending out the data, so my usecase has failed
>> and I will have to fix that.
>>
>> Also, can you please adivse how much is the max data rates you have been
>> able to achieve with one vmxnet3 10G port.
>>
>> Thanks a lot for the advice once again.
>>
>> Regards
>> -Prashant
>>
>>
>> -Original Message-
>> From: Srinivasan J [mailto:srinidpdk at gmail.com]
>> Sent: Sunday, March 09, 2014 12:38 AM
>> To: Prashant Upadhyaya
>> Cc: David Marchand; dev at dpdk.org
>> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>>
>> Prashant,
>> I was also able to hit the issue your hitting using Esxi
>> 5.1.0 evaluation and Fedora 20 X86_64 guest. I was able to fix the issue by
>>
[dpdk-dev] On vmxnet-pmd crash in DPDK 1.6.0r1
I also got the segfault. Setting CONFIG_RTE_EAL_UNBIND_PORTS=y is not ideal
because it would also unbind my VMNET3 management interface.
Does anyone know why the crash is happening? According to the stacktrace,
hw_addrX is 0x0 during eth_vmxnet3_dev_init.
I?m running on ubuntu 12.04LTS on ESXi 5.5 with Intel I350 NIC as the physical
adapter for VMNET3.
Program received signal SIGSEGV, Segmentation fault.
0x004fe0eb in vmxnet3_read_addr (addr=0x0) at
/home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h:139
(gdb) up
#1 0x004fe331 in eth_vmxnet3_dev_init (eth_drv=0x799440
, eth_dev=0x7d5280 ) at
/home/dkan/nyansa/3rd-party/dpdk-1.6.0r1/lib/librte_pmd_v\
mxnet3/vmxnet3_ethdev.c:218
(gdb) print *hw
$12 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968, vendor_id
= 5549, subsystem_device_id = 0, subsystem_vendor_id = 0, adapter_stopped = 0,
perm_addr = "\000\000\\
000\000\000", num_tx_queues = 1 '\001', num_rx_queues = 1 '\001', bufs_per_pkt
= 1 '\001', cur_mtu = 1500, tqd_start = 0x0, rqd_start = 0x0, shared = 0x0,
sharedPA = 0, queueDescPA\
= 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
Thanks in advance.
Dan
On Mar 10, 2014, at 1:00 AM, Prashant Upadhyaya wrote:
> Hi Srini,
>
> Thanks, I could also make it work, thanks to your cue !
>
> Now then, this multi-segment not being supported in vmxnet3 driver is a big
> party-pooper for me. Unfortunately in my usecase, I do indeed make heavy use
> of multisegment buffers for sending out the data, so my usecase has failed
> and I will have to fix that.
>
> Also, can you please adivse how much is the max data rates you have been able
> to achieve with one vmxnet3 10G port.
>
> Thanks a lot for the advice once again.
>
> Regards
> -Prashant
>
>
> -Original Message-
> From: Srinivasan J [mailto:srinidpdk at gmail.com]
> Sent: Sunday, March 09, 2014 12:38 AM
> To: Prashant Upadhyaya
> Cc: David Marchand; dev at dpdk.org
> Subject: Re: [dpdk-dev] Which vmxnet3 pmd is to be used in dpdk 1.6.x?
>
> Prashant,
> I was also able to hit the issue your hitting using Esxi
> 5.1.0 evaluation and Fedora 20 X86_64 guest. I was able to fix the issue by
> setting CONFIG_RTE_EAL_UNBIND_PORTS=y option in
> defconfig_x86_64-default-linuxapp-gcc configuration file.
>
> Issue seen
>
>
> EAL: PCI device :03:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
> EAL: Device is blacklisted, not initializing
> EAL: PCI device :0b:00.0 on NUMA socket -1
> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd
>
> Program received signal SIGSEGV, Segmentation fault.
> eth_vmxnet3_dev_init (eth_drv=, eth_dev=0x754480
> ) at
> /root/source/dpdk-1.6.0r1/lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c:218
> 218 ver = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_VRRS);
> Missing separate debuginfos, use: debuginfo-install glibc-2.18-11.fc20.x86_64
> (gdb) p hw
> $1 = (struct vmxnet3_hw *) 0x7fffd8fc1040
> (gdb) p *hw
> $2 = {hw_addr0 = 0x0, hw_addr1 = 0x0, back = 0x0, device_id = 1968, vendor_id
> = 5549, subsystem_device_id = 0, subsystem_vendor_id = 0,
> adapter_stopped = 0, perm_addr = "\000\000\000\000\000", num_tx_queues = 1
> '\001', num_rx_queues = 1 '\001', bufs_per_pkt = 1 '\001', cur_mtu = 0,
> tqd_start = 0x0, rqd_start = 0x0, shared = 0x0, sharedPA = 0, queueDescPA =
> 0, queue_desc_len = 0, rss_conf = 0x0, rss_confPA = 0, mf_table = 0x0}
> (gdb)
>
> #define VMXNET3_PCI_BAR1_REG_ADDR(hw, reg) \
>((volatile uint32_t *)((char *)(hw)->hw_addr1 + (reg))) #define
> VMXNET3_READ_BAR1_REG(hw, reg) \
>vmxnet3_read_addr(VMXNET3_PCI_BAR1_REG_ADDR((hw), (reg)))
> lib/librte_pmd_vmxnet3/vmxnet3_ethdev.h
>
> Issue not seen after enabling CONFIG_RTE_EAL_UNBIND_PORTS=y
> ==
>
> [root at localhost build]# ./l2fwd -c 0xf -b :03:00.0 -n 1 -- -p 0x6
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Skip lcore 4 (not detected)
> EAL: Skip lcore 5 (not detected)
> EAL: Skip lcore 6 (not detected)
> EAL: Skip lcore 7 (not detected)
> EAL: Skip lcore 8 (not detected)
> EAL: Skip lcore 9 (not detected)
> EAL: Skip lcore 10 (not detected)
> EAL: Skip lcore 11 (not detected)
> EAL: Skip lcore 12 (not detected)
> EAL: Skip lcore 13 (not detected)
> EAL: Skip lcore 14 (not detected)
> EAL: Skip lcore 15 (not detected)
> EAL: Skip lcore 16 (not detected)
> EAL: Skip lcore 17 (not detected)
> EAL: Skip lcore 18 (not detected)
> EAL: Skip lcore 19 (not detected)
> EAL: Skip lcore 20 (not detected)
> EAL: Skip lcore 21 (not detected)
> EAL: Skip lcore 22 (not detected)
> EAL: Skip lcore 23 (not detected)
> EAL: Skip lcore 24 (not detected)
> EAL: Skip lcore 25 (not detected)
> EAL: Skip lcore 26 (not detected)
> EAL: Skip lcore 27 (not detect
[dpdk-dev] Profiling tools for DPDK applications running on VM inside ESXi server
Thanks for the tip. Maybe it?s better to optimize code on bare metal before moving to VM. Dan On Feb 28, 2014, at 6:12 PM, Selvaganapathy Chidambaram wrote: > Hi Dan, > > When we tried, perf did not work in ESX, it only works in KVM. oprofile works > in timer mode in ESX. > > Thanks, > Selvaganapathy.C. > > > On Fri, Feb 28, 2014 at 6:07 PM, Daniel Kan wrote: > Hi, > I?m wondering which profilers are best for profiling DPDK applications > running on VM: perf, oprofile, vtune? The first 2 are open sourced while > vtune is commercial. DPDK apps are running on ubuntu 12.04 and centos 6.5. > Thanks. > > Dan >
[dpdk-dev] Profiling tools for DPDK applications running on VM inside ESXi server
Hi, I?m wondering which profilers are best for profiling DPDK applications running on VM: perf, oprofile, vtune? The first 2 are open sourced while vtune is commercial. DPDK apps are running on ubuntu 12.04 and centos 6.5. Thanks. Dan
[dpdk-dev] Is it possible to get symmetric hash from RSS for a given flow
The symmetric RSK for 5-tuple is a generalized case for 2-tuple src ip/dst ip. You can use the same RSK to get the symmetric load balancing. Dan On Feb 27, 2014, at 5:08 PM, Stefan Baranoff wrote: > Has anyone already applied this logic to IP only load balancing to get > SRC/DST and DST/SRC on the same queue and come up with an RSK value? If not > I'll spend some time with a calculator and see if I can get one. > Thanks, > Stefan > > Sent from my smart phone; people don't make typos, Swype does! > > On Feb 27, 2014 8:04 PM, "Daniel Kan" wrote: > Thanks, Sangjin. This is exactly i?m looking for. Sorry, I should?ve googled > it first. It?s good to know that hash key of repetitive 0x6d5a still yields > good load balancing results. > > Dan > > On Feb 27, 2014, at 4:24 PM, Sangjin Han wrote: > > > Hi, > > > > This is exactly what you want: > > http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf > > > > Sangjin > > > > On Thu, Feb 27, 2014 at 4:22 PM, Daniel Kan wrote: > >> Hi, > >> It appears that the hash computed from RSS is unidirectional. Hence, for a > >> 5-tuple flow, packet in one direction can be routed to a queue that may be > >> different than packet in the other direction. I'm wondering if there is a > >> configuration or mechanism to get a symmetric hash so that all packets for > >> a given flow will be in the same queue. Thanks in advance. > >> > >> Dan >
[dpdk-dev] Is it possible to get symmetric hash from RSS for a given flow
Thanks, Sangjin. This is exactly i?m looking for. Sorry, I should?ve googled it first. It?s good to know that hash key of repetitive 0x6d5a still yields good load balancing results. Dan On Feb 27, 2014, at 4:24 PM, Sangjin Han wrote: > Hi, > > This is exactly what you want: > http://www.ndsl.kaist.edu/~shinae/papers/TR-symRSS.pdf > > Sangjin > > On Thu, Feb 27, 2014 at 4:22 PM, Daniel Kan wrote: >> Hi, >> It appears that the hash computed from RSS is unidirectional. Hence, for a >> 5-tuple flow, packet in one direction can be routed to a queue that may be >> different than packet in the other direction. I'm wondering if there is a >> configuration or mechanism to get a symmetric hash so that all packets for a >> given flow will be in the same queue. Thanks in advance. >> >> Dan
[dpdk-dev] Is it possible to get symmetric hash from RSS for a given flow
Hi, It appears that the hash computed from RSS is unidirectional. Hence, for a 5-tuple flow, packet in one direction can be routed to a queue that may be different than packet in the other direction. I?m wondering if there is a configuration or mechanism to get a symmetric hash so that all packets for a given flow will be in the same queue. Thanks in advance. Dan
[dpdk-dev] What's the performance significance of ixgbe_recv_pkts_bulk_alloc
Hi, While browsing through the ixgbe pmd code, I noticed that there is ixgbe_recv_pkts_bulk_alloc, which can be enabled if the following preconditions are met. * rxq->rx_free_thresh >= RTE_PMD_IXGBE_RX_MAX_BURST * rxq->rx_free_thresh < rxq->nb_rx_desc * (rxq->nb_rx_desc % rxq->rx_free_thresh) == 0 * rxq->nb_rx_desc<(IXGBE_MAX_RING_DESC-RTE_PMD_IXGBE_RX_MAX_BURST) I presume the difference from the normal (non bulk) version has to do with buffer allocation. Can someone please explain the inner working of bulk_alloc and why one may or may not want to enable bulk_alloc mode? I only see bulk_alloc is available for ixgbe driver and not igb and e1000. Why is that? Thanks. Dan
[dpdk-dev] Selecting Linux distribution for DPDK applications: CentOS or Debian
Thanks for the all the feedbacks. Ubuntu 12.04 LTS has 3.8.x kernel and Fedcore 18 has 3.10.x kernel. Both kernels are not compatible with the vmxnet-usermap driver which supports only up to 3.2.x. virtio-net-pmd requires kernel 3.8.x and qemu 1.5 for multi-queue support. Since CentOS?s kernel is on its own path, I?m wondering if anyone has any issue with virtio-net-pmd on CentOS 6.x. Prashant, CentOS is essentially the same as RHEL 6.3 minus the support. Dan On Jan 30, 2014, at 6:06 AM, Hamed khanmirza wrote: > Hi Dan, > > We are also using Debian 7.3, from the DPDK v1.3.1 up to version 1.5.1r1. > it works completely fine. > > regards, > - Hamed > > > On Thu, Jan 30, 2014 at 3:40 PM, Daniel Kaminsky < > daniel.kaminsky at infinitelocality.com> wrote: > >> We're using CentOS 6.5 and Ubuntu 12.04 and don't see any issue with both >> of them. >> >> Regards, >> Daniel Kaminsky >> >> >> On Thu, Jan 30, 2014 at 12:12 PM, Prashant Upadhyaya < >> prashant.upadhyaya at aricent.com> wrote: >> >>> Hi Dan, >>> >>> Intel DPDK release notes(1.5.2) mention the following tested OS -- >>> * Fedora release 18 >>> * Ubuntu* 12.04 LTS >>> * Wind River* Linux* 5 >>> * Red Hat* Enterprise Linux 6.3 >>> * SUSE Enterprise Linux* 11 SP2 >>> >>> I have personally used Fedora 18 and it works fine for me for >>> virtualization including SRIOV and pass through as well as virtio with >> KNI >>> backend. >>> So I am tending to stick to Fedora 18. >>> >>> I don't know why is CentOS not tested and mentioned in release notes. >>> >>> Regards >>> -Prashant >>> >>> -Original Message- >>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Daniel Kan >>> Sent: Thursday, January 30, 2014 2:01 PM >>> To: dev at dpdk.org >>> Subject: [dpdk-dev] Selecting Linux distribution for DPDK applications: >>> CentOS or Debian >>> >>> I'm deciding between Debian 7.3 (3.2.0 kernel) and CentOS 6.5 (2.6.32 >>> kernel) for production. I'm wondering if anyone has recommendation. We >> run >>> the DPDK application in a virtualized environment. Currently, we >> configure >>> NICs in pass-through mode which gives the best performance. In the >> future, >>> we plan to use DPDK with paravirtualized nics (eg. vmxnet3-usermap). >> Thanks. >>> >>> Dan >>> >>> >>> >>> >>> >>> >> === >>> Please refer to http://www.aricent.com/legal/email_disclaimer.html >>> for important disclosures regarding this electronic communication. >>> >>> >> === >>> >>
[dpdk-dev] Selecting Linux distribution for DPDK applications: CentOS or Debian
I?m deciding between Debian 7.3 (3.2.0 kernel) and CentOS 6.5 (2.6.32 kernel) for production. I?m wondering if anyone has recommendation. We run the DPDK application in a virtualized environment. Currently, we configure NICs in pass-through mode which gives the best performance. In the future, we plan to use DPDK with paravirtualized nics (eg. vmxnet3-usermap). Thanks. Dan
[dpdk-dev] [PATCH v3] app/testpmd: fix RSS rx by setting mq_mode
The mq_mode was not set when rxq is > 0; it's defaulted to ETH_MQ_RX_NONE.
As a result, RSS remains inactive. The fix is to set mq_mode to ETH_MQ_RX_RSS
when hf is non-zero.
This bug was introduced by commit 243db2ddee3094a2cb39fdd4b17e26df4e7735e1
igb/ixgbe: ETH_MQ_RX_NONE should disable RSS
Signed-off-by: Daniel Kan
---
Enable RSS even if rxq=1 based on comment from Maxime Leroy.
app/test-pmd/testpmd.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b11eb2e..42e1fdb 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1546,6 +1546,9 @@ init_port_config(void)
if (nb_rxq > 0) {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_hf;
+ if (rss_hf != 0) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ }
} else {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
--
1.7.9.5
[dpdk-dev] [PATCH v3]
The mq_mode was not set when rxq is > 0; it's defaulted to ETH_MQ_RX_NONE.
As a result, RSS remains inactive. The fix is to set mq_mode to ETH_MQ_RX_RSS
when hf is non-zero.
This bug was introduced by commit 243db2ddee3094a2cb39fdd4b17e26df4e7735e1
igb/ixgbe: ETH_MQ_RX_NONE should disable RSS
Signed-off-by: Daniel Kan
---
Enable RSS even if rxq=1 based on comment from Maxime Leroy.
app/test-pmd/testpmd.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b11eb2e..42e1fdb 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1546,6 +1546,9 @@ init_port_config(void)
if (nb_rxq > 0) {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_hf;
+ if (rss_hf != 0) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ }
} else {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
--
1.7.9.5
[dpdk-dev] Any benefit of using DPDK's makefiles instead of using your own and linking against DPDK library
I think in general, developers are pretty particular about their makefiles and build framework. This is especially true if you have an existing application; migrating it to use dpdk?s build framework is rather cumbersome and likely a non-starter for bigger complex projects. Hence, separating application build from dpdk build will be very useful in many situations. The main trick is to extract the appropriate compilation flags that can be readily consumed by the application build system. As an application developer, nothing is simper than just having some header files, a library file, and some necessary compilation flags. If tools like pkg-config or others can facilitate that process, then we could consider it. But even a simple script that can be give me flags that I can provide to my own build system would already suffice. On a similar note, is there any particular reason why many macros such as RTE_MACHINE_XXX are passed as flags to the compiler instead of being defined in rte_config.h? Thanks Dan On Jan 14, 2014, at 5:55 AM, Thomas Monjalon wrote: > Hello Venky, > > 14/01/2014 14:22, Venkatesan, Venky: >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas Monjalon >>> 14/01/2014 08:02, Daniel Kan: >>>> I already have existing makefiles for my current application. I would >>>> like to integrate dpdk into the application. ?m wondering if there is >>>> any benefit to use dpdk?s makefiles instead of using your own makefile >>>> and linking against the library (e.g. libintel_dpdk.a). Thanks. >>> >>> DPDK makefiles have 2 benefits: >>> - provide a framework >>> - automatically set CFLAGS and LDFLAGS according to your configuration >>> >>> If you don't need a framework, I think it's better to extract compilation >>> flags with something like pkg-config. >>> http://freedesktop.org/wiki/Software/pkg-config >>> A patch for a such feature would be welcome :) >> >> One other thing to think about - as we add more functionality into DPDK >> (e.g. new libraries for other packet functions), we integrate them into >> the DPDK framework. If you extract compilation flags and setup your own >> makefile, you would have to do this re-integration every time you want to >> pick up a new release. The same applies to newer files added etc. etc. >> That is the downside. > > I disagree. > If the Makefile of the application, use a DPDK script or pkg-config to read > the flags from a generated file, the integration is done only once. > This guide explains the logic and how to implement it: > http://people.freedesktop.org/~dbn/pkg-config-guide.html > > -- > Thomas
[dpdk-dev] [PATCH v2] app/testpmd: fix RSS by setting mq_mode
Maxime, Thanks for your comment. If we want testpmd to have RSS always enabled out of box (note that rxq=1), then your suggestion makes sense. I?m new to dpdk so I was trying to preserve the current behavior. Now looking back, I guess that?s why we have an explicit disable-rss option. If I don?t hear any resistance by tomorrow, I will submit a new patch. Thanks. Dan PS. I don?t see port_rss_hash_conf_show in config.c, at not in 1.5.1 branch. Which branch is this in? On Jan 14, 2014, at 9:42 AM, Maxime Leroy wrote: > Hello, > > Thanks for your patch fixing the regression introduced by my commit > (igb/ixgbe: ETH_MQ_RX_NONE should disable RSS). > > I have one comment about your fix. I don't think there are any reasons > to not enable RSS with only one RX queue in testpmd. > > RSS is mainly used in testpmd to spread traffic on the different rx queues. > But you can use RSS with one rx queue for debbugging purpose. > For example, you can use the rxonly forward engine of the testpmd to display > the RSS hash. (see pkt_burst_receive in app/test-pmd/rxonly.c) > > An other issue about not enabling RSS with 1 queue, when you use the command > show_rss_key in the testpmd, this one will display that the RSS is enabled. > (because rss_hf != 0; see port_rss_hash_conf_show function in > app/test-pmd/config.c) > > Do you agree with my analyze ? > > Thanks. > > -- > Maxime Leroy
[dpdk-dev] Any benefit of using DPDK's makefiles instead of using your own and linking against DPDK library
I already have existing makefiles for my current application. I would like to integrate dpdk into the application. ?m wondering if there is any benefit to use dpdk?s makefiles instead of using your own makefile and linking against the library (e.g. libintel_dpdk.a). Thanks. Dan
[dpdk-dev] [PATCH v2] app/testpmd: fix RSS by setting mq_mode
The mq_mode was not set when rxq is > 1; it's defaulted to ETH_MQ_RX_NONE.
As a result, RSS remains inactive. The fix is to set mq_mode to ETH_MQ_RX_RSS
when rxq is > 1 and hf is non-zero.
This bug was introduced by commit 243db2ddee3094a2cb39fdd4b17e26df4e7735e1
igb/ixgbe: ETH_MQ_RX_NONE should disable RSS
Signed-off-by: Daniel Kan
---
Updated commit log in accordance with dpdk guidelines
app/test-pmd/testpmd.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b11eb2e..355db0f 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1546,6 +1546,9 @@ init_port_config(void)
if (nb_rxq > 0) {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_hf;
+ if (nb_rxq > 1 && rss_hf != 0) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ }
} else {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
--
1.7.9.5
[dpdk-dev] [PATCH] app/testpmd: fix RSS rx by setting mq_mode
---
app/test-pmd/testpmd.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index b11eb2e..355db0f 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -1546,6 +1546,9 @@ init_port_config(void)
if (nb_rxq > 0) {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_hf;
+ if (nb_rxq > 1 && rss_hf != 0) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ }
} else {
port->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
--
1.7.9.5
[dpdk-dev] Unable to get RSS to work in testpmd and load balancing question
Thanks, Sy Jong. I couldn?t reproduce your outcome on dpdk 1.5.1 with ixgbe. As I sent in the earlier email, rxmode.mq_mode is defaulted to 0 (i.e. ETH_MQ_RX_NONE); it should be set to ETH_MQ_RX_RSS. Dan On Jan 9, 2014, at 6:07 PM, Choi, Sy Jong wrote: > > Hi Dan, > > I have tested with 6 flows with identical ip address, but varies UDP port > number. I can see both queues with traffic. > Using the following command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i -rss-udp --portmask=0x03 --nb-cores=4 > --rxq=2 --txq=2 > > > I have started with RSS IPv4, which is enabled by default. > The critical part is the traffic, since I only 2 queues, I am sending 6 flows > with different IP addresses in order to see the flow got distributed evenly. > Or else you might see only 1 queues if you have 2 flows they might load to a > single queue only. > > My Command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=0x03 --nb-cores=4 --rxq=2 > --txq=2 > - Using 4 cores > - Rxq = 2 for each port, so 4 queues to 4 cores. > > > > testpmd> show port stats all > > NIC statistics for port 0 > RX-packets: 6306519648RX-errors: 757945685RX-bytes: > 309383840254 > TX-packets: 132592678TX-errors: 0TX-bytes: > 8485925376 > > Stats reg 0 RX-packets: 2556150208RX-errors: 0RX-bytes: > 116477417471 > Stats reg 1 RX-packets: 3750369440RX-errors: 0RX-bytes: > 192906422783 > Stats reg 2 RX-packets: 0RX-errors: 0RX-bytes: >0 > . > . > . > Stats reg 15 RX-packets: 0RX-errors: 0RX-bytes: >0 > > > NIC statistics for port 1 > RX-packets: 132594048RX-errors: 13825889RX-bytes: > 8486020288 > TX-packets: 6306522739TX-errors: 0TX-bytes: > 231983528894 > > Stats reg 0 RX-packets: 83615783RX-errors: 0RX-bytes: > 5351410624 > Stats reg 1 RX-packets: 48978265RX-errors: 0RX-bytes: > 3134609664 > Stats reg 2 RX-packets: 0RX-errors: 0RX-bytes: >0 > . > . > . > Stats reg 15 RX-packets: 0RX-errors: 0RX-bytes: >0 > > testpmd> > > > > > My Command:- > sudo ./app/testpmd -c 0x1f -n 4 -- -i --portmask=0x03 --nb-cores=4 --rxq=2 > --txq=2 > - Using 4 cores > - Rxq = 2 for each port, so 4 queues to 4 cores. > > I use this command to map the queue statistic. > testpmd> set stat_qmap rx 0 0 0 > testpmd> set stat_qmap rx 0 1 1 > testpmd> set stat_qmap rx 1 0 0 > testpmd> set stat_qmap rx 1 1 1 > testpmd> start > io packet forwarding - CRC stripping disabled - packets/burst=16 > nb forwarding cores=2 - nb forwarding ports=2 > RX queues=2 - RX desc=128 - RX free threshold=0 > RX threshold registers: pthresh=8 hthresh=8 wthresh=4 > TX queues=2 - TX desc=512 - TX free threshold=0 > TX threshold registers: pthresh=36 hthresh=0 wthresh=0 > TX RS bit threshold=0 - TXQ flags=0x0 > > testpmd> show port stats all > > > > Regards, > Choi, Sy Jong > Platform Application Engineer > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Dan Kan > Sent: Wednesday, January 08, 2014 3:25 PM > To: dev at dpdk.org > Subject: [dpdk-dev] Unable to get RSS to work in testpmd and load balancing > question > > I'm evaluating DPDK using dpdk-1.5.1r1. I have been playing around with the > test-pmd sample app. I'm having a hard time to get RSS to work. I have a > 2-port 82599 Intel X540-DA2 NIC. I'm running the following command to start > the app. > > sudo ./testpmd -c 0x1f -n 2 -- -i --portmask=0x3 --nb-cores=4 --rxq=4 > --txq=4 > > I have a packet generator that sends udp packets with various src IP. > According testpmd, I'm only receiving packets in port 0's queue 0. Packets > are not going into any other queues. I have attached the output from testpmd. > > > --- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 1/Queue= 0 > --- > RX-packets: 100TX-packets: 100TX-dropped: > 0 > -- Forward statistics for port 0 > -- > RX-packets: 100RX-dropped: 0 RX-total: 100 > TX-packets: 0 TX-dropped: 0 TX-total: 0 > > > > -- Forward statistics for port 1 > -- > RX-packets: 0 RX-dropped: 0 RX-total: 0 > TX-packets: 100TX-dropped: 0 TX-total: 100 > > --
[dpdk-dev] Query about this NIC
I?m using X520-DA2 Server Adapter E10G42BTDA PCIe Dual-Port 2xSFP+ to evaluate DPDK. It works fine as far as i can tell. Dan On Jan 9, 2014, at 8:06 AM, Prashant Upadhyaya wrote: > Hi, > > No, I have not used X520-T2 before. > > If anybody on the list has used the above successfully, please advise. > > Jayant, I will await your advice. > > Regards > -Prashant > > > -Original Message- > From: Jayakumar, Muthurajan [mailto:muthurajan.jayakumar at intel.com] > Sent: Thursday, January 09, 2014 9:32 PM > To: Prashant Upadhyaya; dev at dpdk.org > Subject: RE: Query about this NIC > > Hi Prashant, > Thanks for using Intel DPDK. > Intel DPDK supports X520-T2 (previously code named "Iron Pond") > > Will find out the differences between X520-T2 and X520-DA2 Server Adapter > E10G42BTDA PCIe Dual-Port 2xSFP+ Copper 10GSFP+Cu Low-Profile > > BTW, have you used X520-T2? > > Thank you, > M Jay > > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant Upadhyaya > Sent: Thursday, January 09, 2014 7:39 AM > To: dev at dpdk.org > Subject: [dpdk-dev] Query about this NIC > > Hi, > > Would the following NIC work with DPDK - > > Intel 10-Gigabit Ethernet X520-DA2 Server Adapter E10G42BTDA PCIe Dual-Port > 2xSFP+ Copper 10GSFP+Cu Low-Profile > > Regards > -Prashant > > > > > > === > Please refer to http://www.aricent.com/legal/email_disclaimer.html > for important disclosures regarding this electronic communication. > === > > > > > === > Please refer to http://www.aricent.com/legal/email_disclaimer.html > for important disclosures regarding this electronic communication. > ===
[dpdk-dev] Unable to get RSS to work in testpmd and load balancing question
The problem appears to be that rxmode.mq_mode was never set to ETH_MQ_RX_RSS in testpmd.c; it?s initialized to 0. There should probably be a configuration for that, or should be set when rxq > 1. Dan On Jan 8, 2014, at 3:24 PM, Dan Kan wrote: > I'm evaluating DPDK using dpdk-1.5.1r1. I have been playing around with the > test-pmd sample app. I'm having a hard time to get RSS to work. I have a > 2-port 82599 Intel X540-DA2 NIC. I'm running the following command to start > the app. > > sudo ./testpmd -c 0x1f -n 2 -- -i --portmask=0x3 --nb-cores=4 --rxq=4 --txq=4 > > I have a packet generator that sends udp packets with various src IP. > According testpmd, I'm only receiving packets in port 0's queue 0. Packets > are not going into any other queues. I have attached the output from testpmd. > > > --- Forward Stats for RX Port= 0/Queue= 0 -> TX Port= 1/Queue= 0 --- > RX-packets: 100TX-packets: 100TX-dropped: 0 > > -- Forward statistics for port 0 -- > RX-packets: 100RX-dropped: 0 RX-total: 100 > TX-packets: 0 TX-dropped: 0 TX-total: 0 > > > -- Forward statistics for port 1 -- > RX-packets: 0 RX-dropped: 0 RX-total: 0 > TX-packets: 100TX-dropped: 0 TX-total: 100 > > > +++ Accumulated forward statistics for all ports+++ > RX-packets: 100RX-dropped: 0 RX-total: 100 > TX-packets: 100TX-dropped: 0 TX-total: 100 > > > On a separate note, I also find that the CPU utilization using 1 forwarding > core for 2 ports seems to be better (in the aggregate sense) than using 2 > forwarding cores for 2 ports. Running at 10gbps line rate of pktlen=400, with > 1 core, the core's utilization is 40%. With 2 cores, each core's utilization > would 30%, giving an aggregate of 60%. > > I have a use case of only doing rxonly packet processing. From my initial > test, it seems that it's more efficient to have a single core read packets > from both ports, and distribute the packet using rte_ring instead of having > each core read from its port. The rte_eth_rx operations appear to be much CPU > intensive than rte_ring_dequeue operations. > > Thanks in advance. > > Dan
