After packet classification, packets are queued in to batches depending on the matching netdev flow. Thereafter each batch is processed to execute the related actions. This becomes particularly inefficient if there are few packets in each batch as rte_eth_tx_burst() incurs expensive MMIO writes.
This patch series implements intermediate queue for DPDK and vHost User ports. Packets are queued and burst when the packet count exceeds threshold. Also drain logic is implemented to handle cases where packets can get stuck in the tx queues at low rate traffic conditions. Care has been taken to see that latency is well with in the acceptable limits. Testing shows significant performance gains with this implementation. This path series combines the earlier 2 patches posted below. DPDK patch: https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/331039.html vHost User patch: https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332271.html Also this series proposes to disable the retries on vHost User ports and make it configurable via ovsdb.(controversial?) Performance Numbers with intermediate queue: DPDK ports =========== Throughput for P2P scenario, for two 82599ES 10G port with 64 byte packets Number flows MASTER With PATCH ====== ========= ========= 10 10727283 13393844 32 7042253 11228799 50 7515491 9607791 100 5838699 9430730 500 5285066 7845807 1000 5226477 7135601 Latency test MASTER ======= Pkt size min(ns) avg(ns) max(ns) 512 4,631 5,022 309,914 1024 5,545 5,749 104,294 1280 5,978 6,159 45,306 1518 6,419 6,774 946,850 PATCH ===== Pkt size min(ns) avg(ns) max(ns) 512 4,711 5,064 182,477 1024 5,601 5,888 701,654 1280 6,018 6,491 533,037 1518 6,467 6,734 312,471 vHost User ports ================== Throughput for PV scenario, with 64 byte packets Number flows MASTER With PATCH ======== ========= ========= 10 5945899 7833914 32 3872211 6530133 50 3283713 6618711 100 3132540 5857226 500 2964499 5273006 1000 2931952 5178038 Latency test. MASTER ======= Pkt size min(ns) avg(ns) max(ns) 512 10,011 12,100 281,915 1024 7,870 9,313 193,116 1280 7,862 9,036 194,439 1518 8,215 9,417 204,782 PATCH ======= Pkt size min(ns) avg(ns) max(ns) 512 10,492 13,655 281,538 1024 8,407 9,784 205,095 1280 8,399 9,750 194,888 1518 8,367 9,722 196,973 Performance number reported by Eelco Chaudron <eechaudro at redhat.com> at https://mail.openvswitch.org/pipermail/ovs-dev/2017-June/333949.html https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332271.html https://mail.openvswitch.org/pipermail/ovs-dev/2017-April/331039.html ----------------------------------- v1->v2 * Rebased on master due to HW offload changes. * Introduced union for packet count and buffers and changed the variable names appropriately. * No functional change changes. Bhanuprakash Bodireddy (6): netdev: Add netdev_txq_flush function. netdev-dpdk: Add netdev_dpdk_txq_flush function. netdev-dpdk: Add intermediate queue support. dpif-netdev: Flush the packets in intermediate queue. netdev-dpdk: Add netdev_dpdk_vhost_txq_flush function. netdev-dpdk: Enable intermediate queue for vHost User port. lib/dpif-netdev.c | 49 ++++++++++++- lib/netdev-bsd.c | 1 + lib/netdev-dpdk.c | 194 ++++++++++++++++++++++++++++++++++++++++++-------- lib/netdev-dummy.c | 1 + lib/netdev-linux.c | 1 + lib/netdev-provider.h | 8 +++ lib/netdev-vport.c | 2 +- lib/netdev.c | 9 +++ lib/netdev.h | 1 + 9 files changed, 235 insertions(+), 31 deletions(-) -- 2.4.11 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
