On 17/02/2020 02:47, Xiaolong Ye wrote: > This doc describes how to enable DPDK on Openwrt in both virtual and > physical x86 environment. > > Signed-off-by: Xiaolong Ye <xiaolong...@intel.com> > Acked-by: John McNamara <john.mcnam...@intel.com> > --- > > V6 changes: > > 1. addressed review comments raised by Thomas > > V5 changes: > > 1. improve the doc's grammar and wording according to John's > suggestions. > > V4 changes: > > 1. add release notes > > V3 changes: > > 1. emphasize target select in `OpenWrt configuration` section > > V2 changes: > > 1. add meson build steps for dpdk > 2. replace steps about build openwrt and running dpdk application with > links > > > doc/guides/howto/index.rst | 1 + > doc/guides/howto/openwrt.rst | 163 +++++++++++++++++++++++++ > doc/guides/rel_notes/release_20_02.rst | 5 + > 3 files changed, 169 insertions(+) > create mode 100644 doc/guides/howto/openwrt.rst > > diff --git a/doc/guides/howto/index.rst b/doc/guides/howto/index.rst > index a4c131652..5a97ea508 100644 > --- a/doc/guides/howto/index.rst > +++ b/doc/guides/howto/index.rst > @@ -19,3 +19,4 @@ HowTo Guides > packet_capture_framework > telemetry > debug_troubleshoot > + openwrt > diff --git a/doc/guides/howto/openwrt.rst b/doc/guides/howto/openwrt.rst > new file mode 100644 > index 000000000..6081f057b > --- /dev/null > +++ b/doc/guides/howto/openwrt.rst > @@ -0,0 +1,163 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2019 Intel Corporation. > + > +Enable DPDK on OpenWrt > +====================== > + > +This document describes how to enable Data Plane Development Kit (DPDK) on > +OpenWrt in both a virtual and physical x86 environment. > + > +Introduction > +------------ > + > +The OpenWrt project is a well-known source-based router OS which provides a > +fully writable filesystem with package management. > + > +Build OpenWrt > +------------- > + > +You can obtain OpenWrt image through https://downloads.openwrt.org/releases. > +To fully customize your own OpenWrt, it is highly recommended to build it > from > +the source code. You can clone the OpenWrt source code as follows: > + > +.. code-block:: console > + > + git clone https://git.openwrt.org/openwrt/openwrt.git > + > +OpenWrt configuration > +~~~~~~~~~~~~~~~~~~~~~ > + > +* Select ``x86`` in ``Target System`` > +* Select ``x86_64`` in ``Subtarget`` > +* Select ``Build the OpenWrt SDK`` for cross-compilation environment > +* Select ``Use glibc`` in ``Advanced configuration options (for developers)`` > + then ``ToolChain Options`` and ``C Library implementation`` > + > +Kernel configuration > +~~~~~~~~~~~~~~~~~~~~ > + > +The following configurations should be enabled: > + > +* ``CONFIG_VFIO_IOMMU_TYPE1=y`` > +* ``CONFIG_VFIO_VIRQFD=y`` > +* ``CONFIG_VFIO=y`` > +* ``CONFIG_VFIO_NOIOMMU=y`` > +* ``CONFIG_VFIO_PCI=y`` > +* ``CONFIG_VFIO_PCI_MMAP=y`` > +* ``CONFIG_HUGETLBFS=y`` > +* ``CONFIG_HUGETLB_PAGE=y`` > +* ``CONFIG_PROC_PAGE_MONITOR=y`` > +
My 2c is that this is only ok as a short term solution. However we really ought to be aiming for upstream support in OpenWRT, right? For instance, why would enabling VFIO in OpenWRT necessitate a kernel rebuild. We have had Intel IOMMU's for more than 10 years, why wouldn't this be enabled by default? I imagine hugepages have been around a similar period. > +Build steps > +~~~~~~~~~~~ > + > +For detailed OpenWrt build steps and prerequisites, please refer to the > +`OpenWrt build guide > +<https://openwrt.org/docs/guide-developer/build-system/use-buildsystem>`_. > + > +After the build is completed, you can find the images and SDK in > +``<OpenWrt Root>/bin/targets/x86/64-glibc/``. > + > + > +DPDK Cross Compilation for OpenWrt > +---------------------------------- > +Pre-requisites > +~~~~~~~~~~~~~~ > + > +NUMA is required to run DPDK in x86. > + > +.. note:: > + > + For compiling the NUMA lib, run ``libtool --version`` to ensure the > libtool > + version >= 2.2, otherwise the compilation will fail with errors. > + > +.. code-block:: console > + > + git clone https://github.com/numactl/numactl.git > + cd numactl > + git checkout v2.0.13 -b v2.0.13 > + ./autogen.sh > + autoconf -i > + export PATH=<OpenWrt > SDK>/glibc/openwrt-sdk-x86-64_gcc-8.3.0_glibc.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-8.3.0_glibc/bin/:$PATH > + ./configure CC=x86_64-openwrt-linux-gnu-gcc --prefix=<OpenWrt SDK > toolchain dir> > + make install > + > +The numa header files and lib file is generated in the include and lib folder > +respectively under <OpenWrt SDK toolchain dir>. Why not an upstream numactl package in OpenWRT, that just installs the shared libraries and development headers instead of these gymnastics? > +Build DPDK > +~~~~~~~~~~ > + > +To cross compile with meson build, you need to write a customized cross file > +first. > + > +.. code-block:: console > + > + [binaries] > + c = 'x86_64-openwrt-linux-gcc' > + cpp = 'x86_64-openwrt-linux-cpp' > + ar = 'x86_64-openwrt-linux-ar' > + strip = 'x86_64-openwrt-linux-strip' > + > + meson builddir --cross-file openwrt-cross > + ninja -C builddir > + > +.. note:: > + > + For compiling the igb_uio with the kernel version used in target machine, > + you need to explicitly specify kernel_dir in meson_options.txt. As above, why not have an upstream DPDK package in OpenWRT, that just installs the shared libraries and development headers instead of these gymnastics? > +Running DPDK application on OpenWrt > +----------------------------------- > + > +Virtual machine > +~~~~~~~~~~~~~~~ > + > +* Extract the boot image > + > +.. code-block:: console > + > + gzip -d openwrt-x86-64-combined-ext4.img.gz > + > +* Launch Qemu > + > +.. code-block:: console > + > + qemu-system-x86_64 \ > + -cpu host \ > + -smp 8 \ > + -enable-kvm \ > + -M q35 \ > + -m 2048M \ > + -object > memory-backend-file,id=mem,size=2048M,mem-path=/tmp/hugepages,share=on \ > + -drive file=<Your OpenWrt images > folder>/openwrt-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 \ > + -device ide-hd,drive=d0,bus=ide.0 \ > + -net nic,vlan=0 \ > + -net nic,vlan=1 \ > + -net user,vlan=1 \ > + -display none \ > + > + > +Physical machine > +~~~~~~~~~~~~~~~~ > + > +You can use the ``dd`` tool to write the OpenWrt image to the drive you > +want to write the image on. > + > +.. code-block:: console > + > + dd if=openwrt-18.06.1-x86-64-combined-squashfs.img of=/dev/sdX > + > +Where sdX is name of the drive. (You can find it though ``fdisk -l``) > + > +Running DPDK > +~~~~~~~~~~~~ > + > +More detailed info about how to run a DPDK application please refer to > +``Running DPDK Applications`` section of :ref:`the DPDK documentation > <linux_gsg>`. > + > +.. note:: > + > + You need to install pre-built NUMA libraries (including soft link) > + to /usr/lib64 in OpenWrt. > diff --git a/doc/guides/rel_notes/release_20_02.rst > b/doc/guides/rel_notes/release_20_02.rst > index 78dab7cfc..25df894a9 100644 > --- a/doc/guides/rel_notes/release_20_02.rst > +++ b/doc/guides/rel_notes/release_20_02.rst > @@ -229,6 +229,11 @@ New Features > Enhanced the compression performance tool by adding a cycle-count mode > which can be used to help measure and tune hardware and software PMDs. > > +* **Added OpenWrt howto guide.** > + > + Added document describes how to enable DPDK on OpenWrt in both virtual and > + physical machine. > + > > Removed Items > ------------- >
