Hi Maxim,
please see inline
On 8/30/2026 9:30 AM, Maxim Fedyarov wrote:
Hello Vladimir!
I'm currently working on some project with DPDK and Intel E810. This
is my first experience with DPDK and for now I'm running into some
problem and have a question. I saw your commits in DPDK and thought it
would be great to talk directly to the developers. I realize that as
an Intel customer it would be more appropriate for me to contact
technical support first, so feel free to direct me to them.
Welcome to the DPDK community! The [email protected] mailing list is a good
place for this kind of questions with plenty of devs there.
For some reason I can't get more than 86 Mpps with 78 bytes packets on
testpmd with macswap forward mode when the theoretical maximum is
around 128 Mpps on 100Gb. It's strange, because in NIC performance
report
(https://fast.dpdk.org/doc/perf/DPDK_22_07_Intel_NIC_performance_report.pdf)
Test#5 shows 116 Mpps for 64 bytes packets.
I try to troubleshoot it on my own DPDK application "protector-dpdk"
with some CPU work with packets and find out that when i use 10 cores
and get ~86 Mpps than my average batch of RX packets is 64. But if I
add another 2 threads, then it decreases sharply to ~12 packets in the
RX batch.
iperf one first line shows this
6.13% protector-dpdk librte_eal.so.26.0 [.] rte_delay_us_block
I try to find information but I don't find anyone else to encounter
this. Maybe you could suggest what else I could check ?
I run with this command:
# ./dpdk-testpmd -l 0-4 -n 6 -a 12:00.0 --socket-mem=4096 --
--burst=64 --txq=4 --rxq=4 --rxd=512 --txd=512 --mbcache=512 -i -a
--rss-ip --port-topology=loop --nb-cores 4 --forward-mode=macswap
--eth-peer=0,94:6d:ae:b9:ad:c0
and get RX-missed after 86 Mpps:
######################## NIC statistics for port 0
########################
RX-packets: 2542840830 <tel:(254)%20284-0830> RX-missed: 583927799
RX-bytes: 188171278008
RX-errors: 0
RX-nombuf: 0
TX-packets: 2389511082 TX-errors: 0 TX-bytes: 176823803890
Throughput (since last show)
Rx-pps: 95506607 Rx-bps: 56539950368
Tx-pps: 87209523 Tx-bps: 51628035504
############################################################################
Adding more cores doesn't change the situation. I also played with
burst, rxd,txd, mbcache and nothing get me more than ~86 Mpps.
DPDK 25.11 is builded with RTE_NET_INTEL_USE_16BYTE_DESC. By the way,
when i read the docs, i found only
https://edc.intel.com/content/www/us/en/design/products/ethernet/config-guide-e810-dpdk/high-performance-of-small-packets-on-100g-nic-use-16-bytes-rx-descriptor-size/
when this define has old name CONFIG_RTE_LIBRTE_ICE_16BYTE_RX_DESC.
I have following configuration:
1) OS: Ubuntu 24.04
Firstly i test it on Ubuntu 22.02 and the results there were worse
2) CPU: Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
3) Grub
GRUB_CMDLINE_LINUX_DEFAULT="isolcpus=0-23 nohz_full=0-23
rcu_nocbs=0-23 default_hugepagesz=1G hugepagesz=1G hugepages=40
intel_iommu=on iommu=pt idle=poll nmi_watchdog=0 audit=0 nosoftlockup
processor.max_cstate=0 intel_idle.max_cstate=0 hpet=disable mce=off
tsc=reliable numa_balancing=disable"
4) Firmware
# ethtool -i ens2f1np1
driver: ice
version: 6.8.0-136-generic
firmware-version: 5.01 0x80021f8b 1.4031.0
5) PCI 3.0
LnkSta: Speed 8GT/s (downgraded), Width x16
I hope this is not a bottleneck for 100Gb
That is the problem. You hit the PCIe bottleneck. With PCIe 4.0 x16 you
should achieve better results.
6) Two T-rex traffic generators each produced 50 Gb connected to DuT
via switch. And i can see that switch is send 100 Gb
And another question is that I need to do QoS for my project. In my
DPDK application I process packets from one port. And critical service
traffic (like BFD) will also flow through it. So I need to allocate
around 0.01% of bandwidth for this traffic. Do i understand correctly
that if i use vfio-pci and use PF in DPDK application then *i can use
only PFC on layer-2 with PCP value in VLAN-tag*?
As i read in "Intel® Ethernet 800 Series Linux Flow Control
Configuration Guide for RDMA Use Cases" for DSCP-based PFC on E810 i
must use openlldp user-space daemon and it means that it cannot be
used with device binded to vfio-pci ?
I haven't checked it yet, but I'm wondering if maybe if I split my PF
into VF's and apply DSCP-based configuration on PF then will it work
and allocate me my bandwidth?
Yes, you can configure DCB/PFC based on DSCP with kernel driver using
lldptool and work with VF from DPDK. This mode(i.e. DSCP based)
currently is not supported in DPDK ICE PMD. Also please note, ETS
bandwidth granularity is 1%, and, with current implementation, all
non-TC0 traffic will be received by queue0 of your VF. But looks like
this should be fine for your usecase.
Alternatively, if you decide to use PF, you can workaround this by
remapping dscp->pcp on a switch.
Thank you for your time. I would appreciate any response from you.
Best regards,
Maxim Fedyarov
--
Regards,
Vladimir
