Upgrading to DPDK 17.05.1 stable release adds new
significant features relevant to OVS, including,
but not limited to:
- tun/tap PMD,
- VFIO hotplug support,
- Generic flow API.
Following changes are applied:
- netdev-dpdk: Changes required by DPDK API modifications.
- doc: Because of DPDK API changes, backward compatibility
with previous DPDK releases will be broken, thus all
relevant documentation entries are updated.
- .travis: DPDK version change from 16.11.1 to 17.05.1.
- rhel/openvswitch-fedora.spec.in: DPDK version change
from 16.11 to 17.05.1
v1->v2: Patch rebase.
v2->v3: Fixed wrong formating after v2 patch rebase.
v3->v4: Minor documentation changes.
Signed-off-by: Michal Weglicki <michalx.wegli...@intel.com>
Reviewed-by: Aaron Conole <acon...@redhat.com>
---
.travis/linux-build.sh | 2 +-
Documentation/faq/releases.rst | 1 +
Documentation/howto/dpdk.rst | 6 +-
Documentation/intro/install/dpdk.rst | 14 +--
Documentation/topics/dpdk/vhost-user.rst | 12 +--
NEWS | 1 +
lib/netdev-dpdk.c | 144 +++++++++++++++++++------------
rhel/openvswitch-fedora.spec.in | 2 +-
tests/dpdk/ring_client.c | 6 +-
9 files changed, 114 insertions(+), 74 deletions(-)
diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh
index f66b534..efccdf1 100755
--- a/.travis/linux-build.sh
+++ b/.travis/linux-build.sh
@@ -80,7 +80,7 @@ fi
if [ "$DPDK" ]; then
if [ -z "$DPDK_VER" ]; then
- DPDK_VER="16.11.2"
+ DPDK_VER="17.05.1"
fi
install_dpdk $DPDK_VER
if [ "$CC" = "clang" ]; then
diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst
index 707834b..2ecc24c 100644
--- a/Documentation/faq/releases.rst
+++ b/Documentation/faq/releases.rst
@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work
with?
2.5.x 2.2
2.6.x 16.07.2
2.7.x 16.11.2
+ 2.8.x 17.05.1
============ =======
Q: I get an error like this when I configure Open vSwitch:
diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst
index af01d3e..1756c8c 100644
--- a/Documentation/howto/dpdk.rst
+++ b/Documentation/howto/dpdk.rst
@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the
application::
When you finish testing, bind the vNICs back to kernel::
- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0
- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0
.. note::
Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved
like so::
- $ $DPDK_DIR/tools/dpdk-devbind.py --status
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status
More information on the dpdkvhostuser ports can be found in
:doc:`/topics/dpdk/vhost-user`.
diff --git a/Documentation/intro/install/dpdk.rst
b/Documentation/intro/install/dpdk.rst
index a05aa1a..20baa63 100644
--- a/Documentation/intro/install/dpdk.rst
+++ b/Documentation/intro/install/dpdk.rst
@@ -40,7 +40,7 @@ Build requirements
In addition to the requirements described in :doc:`general`, building Open
vSwitch with DPDK will require the following:
-- DPDK 16.11
+- DPDK 17.05.1
- A `DPDK supported NIC`_
@@ -69,9 +69,9 @@ Install DPDK
#. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``::
$ cd /usr/src/
- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz
- $ tar xf dpdk-16.11.2.tar.xz
- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2
+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz
+ $ tar xf dpdk-17.05.1.tar.xz
+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1
$ cd $DPDK_DIR
#. (Optional) Configure DPDK as a shared library
@@ -187,8 +187,8 @@ to the VFIO driver::
$ modprobe vfio-pci
$ /usr/bin/chmod a+x /dev/vfio
$ /usr/bin/chmod 0666 /dev/vfio/*
- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1
- $ $DPDK_DIR/tools/dpdk-devbind.py --status
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status
Setup OVS
~~~~~~~~~
@@ -569,7 +569,7 @@ Limitations
The latest list of validated firmware versions can be found in the `DPDK
release notes`_.
-.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16_11.html
+.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_17_05.html
Reporting Bugs
--------------
diff --git a/Documentation/topics/dpdk/vhost-user.rst
b/Documentation/topics/dpdk/vhost-user.rst
index e76da5f..a3d5de3 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in
:ref:`dpdk-vhost-user` or
DPDK sources to VM and build DPDK::
$ cd /root/dpdk/
- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz
- $ tar xf dpdk-16.11.2.tar.xz
- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2
+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz
+ $ tar xf dpdk-17.05.1.tar.xz
+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1
$ export DPDK_TARGET=x86_64-native-linuxapp-gcc
$ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET
$ cd $DPDK_DIR
@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO::
$ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already
mounted
$ modprobe uio
$ insmod $DPDK_BUILD/kmod/igb_uio.ko
- $ $DPDK_DIR/tools/dpdk-devbind.py --status
- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0
+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status
+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0
.. note::
@@ -378,7 +378,7 @@ Sample XML
</disk>
<disk type='dir' device='disk'>
<driver name='qemu' type='fat'/>
- <source dir='/usr/src/dpdk-stable-16.11.2'/>
+ <source dir='/usr/src/dpdk-stable-17.05.1'/>
<target dev='vdb' bus='virtio'/>
<readonly/>
</disk>
diff --git a/NEWS b/NEWS
index d61fc5f..3f58e8b 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,7 @@ Post-v2.7.0
still can be configured via extra arguments for DPDK EAL.
* dpdkvhostuser ports are marked as deprecated. They will be removed
in an upcoming release.
+ * Support for DPDK v17.05.1.
- IPFIX now provides additional counters:
* Total counters since metering process startup.
* Per-flow TCP flag counters.
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index ea17b97..5e767e1 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -22,6 +22,9 @@
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
+#include <linux/virtio_net.h>
+#include <sys/socket.h>
+#include <linux/if.h>
#include <rte_config.h>
#include <rte_cycles.h>
@@ -31,7 +34,7 @@
#include <rte_malloc.h>
#include <rte_mbuf.h>
#include <rte_meter.h>
-#include <rte_virtio_net.h>
+#include <rte_vhost.h>
#include "dirs.h"
#include "dp-packet.h"
@@ -56,6 +59,8 @@
#include "timeval.h"
#include "unixctl.h"
+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM};
+
VLOG_DEFINE_THIS_MODULE(netdev_dpdk);
static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20);
@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = {
},
};
+/*
+ * These callbacks allow virtio-net devices to be added to vhost ports when
+ * configuration has been fully completed.
+ */
+static int new_device(int vid);
+static void destroy_device(int vid);
+static int vring_state_changed(int vid, uint16_t queue_id, int enable);
+static const struct vhost_device_ops virtio_net_device_ops =
+{
+ .new_device = new_device,
+ .destroy_device = destroy_device,
+ .vring_state_changed = vring_state_changed,
+ .features_changed = NULL
+};
+
enum { DPDK_RING_SIZE = 256 };
BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE));
enum { DRAIN_TSC = 200000ULL };
@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk {
dpdk_port_t port_id;
};
-static int netdev_dpdk_class_init(void);
-static int netdev_dpdk_vhost_class_init(void);
+static void netdev_dpdk_destruct(struct netdev *netdev);
+static void netdev_dpdk_vhost_destruct(struct netdev *netdev);
int netdev_dpdk_get_vid(const struct netdev_dpdk *dev);
@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk
*dev);
static bool
is_dpdk_class(const struct netdev_class *class)
{
- return class->init == netdev_dpdk_class_init
- || class->init == netdev_dpdk_vhost_class_init;
+ return class->destruct == netdev_dpdk_destruct
+ || class->destruct == netdev_dpdk_vhost_destruct;
}
/* DPDK NIC drivers allocate RX buffers at a particular granularity, typically
@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev)
if (err) {
VLOG_ERR("vhost-user socket device setup failure for socket %s\n",
dev->vhost_id);
+ goto out;
} else {
fatal_signal_add_file_to_unlink(dev->vhost_id);
VLOG_INFO("Socket %s created for vhost-user port %s\n",
dev->vhost_id, name);
}
+
+ err = rte_vhost_driver_callback_register(dev->vhost_id,
+ &virtio_net_device_ops);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user "
+ "port: %s\n", name);
+ goto out;
+ }
+
+ err = rte_vhost_driver_disable_features(dev->vhost_id,
+ 1ULL << VIRTIO_NET_F_HOST_TSO4
+ | 1ULL << VIRTIO_NET_F_HOST_TSO6
+ | 1ULL << VIRTIO_NET_F_CSUM);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user "
+ "port: %s\n", name);
+ goto out;
+ }
+
+ err = rte_vhost_driver_start(dev->vhost_id);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_start failed for vhost user "
+ "port: %s\n", name);
+ goto out;
+ }
+
err = vhost_common_construct(netdev);
+ if (err) {
+ VLOG_ERR("vhost_common_construct failed for vhost user "
+ "port: %s\n", name);
+ }
+out:
ovs_mutex_unlock(&dpdk_mutex);
VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; "
"please migrate to dpdkvhostuserclient ports.");
@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev *netdev)
ovs_mutex_lock(&dpdk_mutex);
err = vhost_common_construct(netdev);
+ if (err) {
+ VLOG_ERR("vhost_common_construct failed for vhost user client"
+ "port: %s\n", netdev->name);
+ }
ovs_mutex_unlock(&dpdk_mutex);
return err;
}
@@ -2456,12 +2512,9 @@ static void
set_irq_status(int vid)
{
uint32_t i;
- uint64_t idx;
- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) {
- idx = i * VIRTIO_QNUM;
- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0);
- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0);
+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) {
+ rte_vhost_enable_guest_notification(vid, i, 0);
}
}
@@ -2524,7 +2577,7 @@ new_device(int vid)
LIST_FOR_EACH(dev, list_node, &dpdk_list) {
ovs_mutex_lock(&dev->mutex);
if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) {
- uint32_t qp_num = rte_vhost_get_queue_num(vid);
+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM;
/* Get NUMA information */
newnode = rte_vhost_get_numa_node(vid);
@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk
*dev)
return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer);
}
-/*
- * These callbacks allow virtio-net devices to be added to vhost ports when
- * configuration has been fully complete.
- */
-static const struct virtio_net_device_ops virtio_net_device_ops =
-{
- .new_device = new_device,
- .destroy_device = destroy_device,
- .vring_state_changed = vring_state_changed
-};
-
-static void *
-start_vhost_loop(void *dummy OVS_UNUSED)
-{
- pthread_detach(pthread_self());
- /* Put the vhost thread into quiescent state. */
- ovsrcu_quiesce_start();
- rte_vhost_driver_session_start();
- return NULL;
-}
-
static int
netdev_dpdk_class_init(void)
{
@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void)
return 0;
}
-static int
-netdev_dpdk_vhost_class_init(void)
-{
- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER;
-
- /* This function can be called for different classes. The initialization
- * needs to be done only once */
- if (ovsthread_once_start(&once)) {
- rte_vhost_driver_callback_register(&virtio_net_device_ops);
- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4
- | 1ULL << VIRTIO_NET_F_HOST_TSO6
- | 1ULL << VIRTIO_NET_F_CSUM);
- ovs_thread_create("vhost_thread", start_vhost_loop, NULL);
-
- ovsthread_once_done(&once);
- }
-
- return 0;
-}
/* Client Rings */
@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev
*netdev)
"using client socket '%s'",
dev->up.name, dev->vhost_id);
}
+
+ err = rte_vhost_driver_callback_register(dev->vhost_id,
+ &virtio_net_device_ops);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_callback_register failed for "
+ "vhost user client port: %s\n", dev->up.name);
+ goto unlock;
+ }
+
+ err = rte_vhost_driver_disable_features(dev->vhost_id,
+ 1ULL << VIRTIO_NET_F_HOST_TSO4
+ | 1ULL << VIRTIO_NET_F_HOST_TSO6
+ | 1ULL << VIRTIO_NET_F_CSUM);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user "
+ "client port: %s\n", dev->up.name);
+ goto unlock;
+ }
+
+ err = rte_vhost_driver_start(dev->vhost_id);
+ if (err) {
+ VLOG_ERR("rte_vhost_driver_start failed for vhost user "
+ "client port: %s\n", dev->up.name);
+ goto unlock;
+ }
}
err = dpdk_vhost_reconfigure_helper(dev);
@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class =
static const struct netdev_class dpdk_vhost_class =
NETDEV_DPDK_CLASS(
"dpdkvhostuser",
- netdev_dpdk_vhost_class_init,
+ NULL,
netdev_dpdk_vhost_construct,
netdev_dpdk_vhost_destruct,
NULL,
@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class =
static const struct netdev_class dpdk_vhost_client_class =
NETDEV_DPDK_CLASS(
"dpdkvhostuserclient",
- netdev_dpdk_vhost_class_init,
+ NULL,
netdev_dpdk_vhost_client_construct,
netdev_dpdk_vhost_destruct,
netdev_dpdk_vhost_client_set_config,
diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in
index 3a045d3..2bb7102 100644
--- a/rhel/openvswitch-fedora.spec.in
+++ b/rhel/openvswitch-fedora.spec.in
@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel
%endif
%if %{with dpdk}
BuildRequires: libpcap-devel numactl-devel
-BuildRequires: dpdk-devel >= 16.11
+BuildRequires: dpdk-devel >= 17.05.1
Provides: %{name}-dpdk = %{version}-%{release}
%endif
diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c
index b153713..8cc3fb5 100644
--- a/tests/dpdk/ring_client.c
+++ b/tests/dpdk/ring_client.c
@@ -185,15 +185,15 @@ main(int argc, char *argv[])
/* Try dequeuing max possible packets first, if that fails, get the
* most we can. Loop body should only execute once, maximum.
*/
- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) &&
- rx_pkts > 0) {
+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts,
+ rx_pkts, NULL) != 0) && rx_pkts > 0) {
rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring),
PKT_READ_SIZE);
}
if (rx_pkts > 0) {
/* blocking enqueue */
do {
- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts);
+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL);
} while (rslt == -ENOBUFS);
}
}
--
1.8.3.1
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
