add vhost user documentation
fix some minor issues
Signed-off-by: Huawei Xie <huawei.xie at intel.com>
---
doc/guides/sample_app_ug/vhost.rst | 123 ++++++++++++++++++++++++++++---------
1 file changed, 94 insertions(+), 29 deletions(-)
diff --git a/doc/guides/sample_app_ug/vhost.rst
b/doc/guides/sample_app_ug/vhost.rst
index fa53db6..4f28abd 100644
--- a/doc/guides/sample_app_ug/vhost.rst
+++ b/doc/guides/sample_app_ug/vhost.rst
@@ -45,7 +45,7 @@ Background
Virtio networking (virtio-net) was developed as the Linux* KVM
para-virtualized method for communicating network packets
between host and guest.
It was found that virtio-net performance was poor due to context switching and
packet copying between host, guest, and QEMU.
-The following figure shows the system architecture for a virtio- based
networking (virtio-net).
+The following figure shows the system architecture for a virtio-based
networking (virtio-net).
.. _figure_16:
@@ -89,20 +89,34 @@ Sample Code Overview
The DPDK vhost-net sample code demonstrates KVM (QEMU) offloading the
servicing of a Virtual Machine's (VM's)
virtio-net devices to a DPDK-based application in place of the kernel's
vhost-net module.
-The DPDK vhost-net sample code is a simple packet switching application with
the following features:
+The DPDK vhost-net sample code is based on vhost library. Vhost library is
developed for user space ethernet switch to
+easily integrate with vhost functionality.
+
+The vhost library implements the following features:
* Management of virtio-net device creation/destruction events.
-* Mapping of the VM's physical memory into the DPDK vhost-net sample code's
address space.
+* Mapping of the VM's physical memory into the DPDK vhost-net's address
space.
* Triggering/receiving notifications to/from VMs via eventfds.
* A virtio-net back-end implementation providing a subset of virtio-net
features.
+There are two vhost implementations in vhost library, vhost cuse and vhost
user. In vhost cuse, a character device driver is implemented to
+receive and process vhost requests through ioctl messages. In vhost user, a
socket server is created to received vhost requests through
+socket messages. Most of the messages share the same handler routine.
+
+.. note::
+ **Any vhost cuse specific requirement in the following sections will be
emphasized**.
+
+Two impelmentations are turned on and off statically through configure file.
Only one implementation could be turned on. They don't co-exist in current
implementation.
+
+The vhost sample code application is a simple packet switching application
with the following feature:
+
* Packet switching between virtio-net devices and the network interface card,
including using VMDQs to reduce the switching that needs to be performed
in software.
-The following figure shows the architecture of the Vhost sample application.
+The following figure shows the architecture of the Vhost sample application
based on vhost-cuse.
.. _figure_18:
@@ -131,15 +145,19 @@ The example in this section have been validated with the
following distributions
* Fedora* 19
+* Fedora* 20
+
Prerequisites
-------------
This section lists prerequisite packages that must be installed.
-Installing Packages on the Host
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Installing Packages on the Host(vhost cuse required)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The vhost sample code uses the following packages; fuse, fuse-devel, and
kernel- modules-extra.
+The vhost cuse code uses the following packages; fuse, fuse-devel, and
kernel-modules-extra.
+The vhost user code don't rely on those modules as eventfds are already
installed into vhost process through
+unix domain socket.
#. Install Fuse Development Libraries and headers:
@@ -153,6 +171,11 @@ The vhost sample code uses the following packages; fuse,
fuse-devel, and kernel-
yum -y install kernel-modules-extra
+QEMU simulator
+~~~~~~~~~~~~~~
+
+For vhost user, qemu 2.2 is required.
+
Setting up the Execution Environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -202,7 +225,7 @@ In this section, we create a second hugetlbs mount point to
allocate hugepages f
.. code-block:: console
- echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/ nr_hugepages
+ echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
#. Mount hugetlbs at a separate mount point for 2 MB pages:
@@ -251,6 +274,8 @@ at the command line as follows.
Observe that in the example, "-device" and "-netdev" are repeated for two
virtio-net devices.
+For vhost cuse:
+
.. code-block:: console
user at target:~$ qemu-system-x86_64 ... \
@@ -259,15 +284,46 @@ at the command line as follows.
-netdev tap,id=hostnet2,vhost=on,vhostfd=<open fd> \
-device virtio-net-pci, netdev=hostnet2,id=net1
+For vhost user:
+
+.. code-block:: console
+
+ user at target:~$ qemu-system-x86_64 ... \
+ -chardev socket,id=char1,path=<sock_path> \
+ -netdev type=vhost-user,id=hostnet1,chardev=char1 \
+ -device virtio-net-pci,netdev=hostnet1,id=net1 \
+ -chardev socket,id=char2,path=<sock_path> \
+ -netdev type=vhost-user,id=hostnet2,chardev=char2 \
+ -device virtio-net-pci,netdev=hostnet2,id=net2
+
+sock_path is the path for the socket file created by vhost.
Compiling the Sample Code
-------------------------
+#. Compile vhost lib:
+
+ To enable vhost, turn on vhost library in the configure file
config/common_linuxapp.
+
+ .. code-block:: console
+
+ CONFIG_RTE_LIBRTE_VHOST=n
+
+ vhost user is turned on by default in the lib/librte_vhost/Makefile.
+ To enable vhost cuse, uncomment vhost cuse and comment vhost user
manually. In future, a configure will be created for switch between two
implementations.
+
+ .. code-block:: console
+
+ SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_cuse/vhost-net-cdev.c
vhost_cuse/virtio-net-cdev.c vhost_cuse/eventfd_copy.c
+ #SRCS-$(CONFIG_RTE_LIBRTE_VHOST) += vhost_user/vhost-net-user.c
vhost_user/virtio-net-user.c vhost_user/fd_man.c
+
+ After vhost is enabled and the implementation is selected, build the
vhost library.
#. Go to the examples directory:
.. code-block:: console
- export RTE_SDK=/path/to/rte_sdk cd ${RTE_SDK}/examples/vhost-net
+ export RTE_SDK=/path/to/rte_sdk
+ cd ${RTE_SDK}/examples/vhost
#. Set the target (a default target is used if not specified). For example:
@@ -309,13 +365,13 @@ Compiling the Sample Code
cd ${RTE_SDK}/examples/vhost
make
-#. Go to the eventfd_link directory:
+#. Go to the eventfd_link directory(vhost cuse required):
.. code-block:: console
- cd ${RTE_SDK}/examples/vhost-net/eventfd_link
+ cd ${RTE_SDK}/lib/librte_vhost/eventfd_link
-#. Build the eventfd_link kernel module:
+#. Build the eventfd_link kernel module(vhost cuse required):
.. code-block:: console
@@ -324,20 +380,20 @@ Compiling the Sample Code
Running the Sample Code
-----------------------
-#. Install the cuse kernel module:
+#. Install the cuse kernel module(vhost cuse required):
.. code-block:: console
modprobe cuse
-#. Go to the eventfd_link directory:
+#. Go to the eventfd_link directory(vhost cuse required):
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/vhost-net/eventfd_link
+ cd ${RTE_SDK}/lib/librte_vhost/eventfd_link
-#. Install the eventfd_link module:
+#. Install the eventfd_link module(vhost cuse required):
.. code-block:: console
@@ -348,14 +404,22 @@ Running the Sample Code
.. code-block:: console
export RTE_SDK=/path/to/rte_sdk
- cd ${RTE_SDK}/examples/vhost-net
+ cd ${RTE_SDK}/examples/vhost
#. Run the vhost-switch sample code:
+ vhost cuse:
+
.. code-block:: console
user at target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /
mnt/huge -- -p 0x1 --dev-basename usvhost --dev-index 1
+ vhost user: a socket file named usvhost will be created under current
directory. Use its path as the socket path in guest's qemu commandline.
+
+ .. code-block:: console
+
+ user at target:~$ ./build/app/vhost-switch -c f -n 4 --huge-dir /
mnt/huge -- -p 0x1 --dev-basename usvhost
+
.. note::
Please note the huge-dir parameter instructs the DPDK to allocate its
memory from the 2 MB page hugetlbfs.
@@ -364,7 +428,7 @@ Parameters
~~~~~~~~~~
**Basename and Index.**
-The DPDK vhost-net sample code uses a Linux* character device to communicate
with QEMU.
+vhost cuse uses a Linux* character device to communicate with QEMU.
The basename and the index are used to generate the character devices name.
/dev/<basename>-<index>
@@ -477,15 +541,15 @@ QEMU must be executed with specific parameters to:
.. code-block:: console
- user at target:~$ qemu-system-x86_64 ... -device virtio-net-pci,
netdev=hostnet1,id=net1 ...
+ user at target:~$ qemu-system-x86_64 ... -device
virtio-net-pci,netdev=hostnet1,id=net1 ...
* Ensure the guest's virtio-net network adapter is configured with offloads
disabled.
.. code-block:: console
- user at target:~$ qemu-system-x86_64 ... -device virtio-net-pci,
netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_
tso6=off,guest_ecn=off
+ user at target:~$ qemu-system-x86_64 ... -device
virtio-net-pci,netdev=hostnet1,id=net1,csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off
-* Redirect QEMU to communicate with the DPDK vhost-net sample code in place
of the vhost-net kernel module.
+* Redirect QEMU to communicate with the DPDK vhost-net sample code in place
of the vhost-net kernel module(vhost cuse).
.. code-block:: console
@@ -502,8 +566,8 @@ QEMU must be executed with specific parameters to:
The QEMU wrapper (qemu-wrap.py) is a Python script designed to automate
the QEMU configuration described above.
It also facilitates integration with libvirt, although the script may also
be used standalone without libvirt.
-Redirecting QEMU to vhost-net Sample Code
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Redirecting QEMU to vhost-net Sample Code(vhost cuse)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To redirect QEMU to the vhost-net sample code implementation of the vhost-net
API,
an open file descriptor must be passed to QEMU running as a child process.
@@ -516,7 +580,7 @@ an open file descriptor must be passed to QEMU running as a
child process.
.. note::
- This process is automated in the QEMU wrapper script discussed in Section
22.7.3.
+ This process is automated in the QEMU wrapper script discussed in Section
24.7.3.
Mapping the Virtual Machine's Memory
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -533,7 +597,8 @@ In this case, the path passed to the guest should be that
of the 1 GB page huget
.. note::
- This process is automated in the QEMU wrapper script discussed in Section
22.7.3.
+ This process is automated in the QEMU wrapper script discussed in Section
24.7.3.
+ The following two sections only applies to vhost cuse. For vhost-user,
please make corresponding changes to qemu-wrapper script and guest XML file.
QEMU Wrapper Script
~~~~~~~~~~~~~~~~~~~
@@ -562,8 +627,8 @@ which will become the following call to QEMU:
.. code-block:: console
/usr/local/bin/qemu-system-x86_64 -machine pc-i440fx-1.4,accel=kvm,usb=off
-cpu SandyBridge -smp 4,sockets=4,cores=1,threads=1
- -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> -device virtio-net-
pci,netdev=hostnet1,id=net1,
- csum=off,gso=off,guest_tso4=off,gu est_tso6=off,guest_ecn=off -hda <disk
img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
+ -netdev tap,id=hostnet1,vhost=on,vhostfd=<open fd> -device
virtio-net-pci,netdev=hostnet1,id=net1,
+ csum=off,gso=off,guest_tso4=off,guest_tso6=off,guest_ecn=off -hda <disk
img> -m 4096 -mem-path /dev/hugepages -mem-prealloc
Libvirt Integration
~~~~~~~~~~~~~~~~~~~
@@ -630,7 +695,7 @@ To call the QEMU wrapper automatically from libvirt, the
following configuration
emul_path = "/usr/local/bin/qemu-system-x86_64"
- * Configure the "us_vhost_path" variable to point to the DPDK vhost- net
sample code's character devices name.
+ * Configure the "us_vhost_path" variable to point to the DPDK vhost-net
sample code's character devices name.
DPDK vhost-net sample code's character device will be in the format
"/dev/<basename>-<index>".
.. code-block:: xml
@@ -650,7 +715,7 @@ The number of free hugepages can be checked as follows:
.. code-block:: console
- user at target:cat /sys/kernel/mm/hugepages/hugepages-<pagesize> /
nr_hugepages
+ user at target:cat
/sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages
The command above indicates how many hugepages are free to support QEMU's
allocation request.
--
1.8.1.4
