[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
On 2015/8/10 9:11, Zhang, Helin wrote: > >> -Original Message- >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas Monjalon >> Sent: Monday, August 10, 2015 6:38 AM >> To: Xu, Qian Q >> Cc: dev at dpdk.org >> Subject: Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how >> to get DPDK high perf on Intel platform. >> [../..] >>> +1. For Intel? 40G NICs, special configurations should be set before >>> compiling it, >> as follows. **Note**: This is very important:: >>> + >>> + for at least DPDK release 1.8, 2.0 and 2.1, in >> /config/common_linuxapp >>> + CONFIG_RTE_PCI_CONFIG=y >>> + CONFIG_RTE_PCI_EXTENDED_TAG=?on? >> Please insert it in a i40e doc instead of here. Then you can reference it. > Good idea! Yes, agree, actually, for RRC(fm10k), this configure is meaningless. Thanks, Michael > Regards, > Helin > >
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
Hi Helin, Your comment is unreadable. Please remove useless lines of your reply.
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
Sorry for repeat! I should have removed the useless pieces! > +Hardware platform Network Interface Card Essential requirements: > + > > + > +1. Get an high end Intel(r) NIC, e.g: Intel(r) XL710. > + > +2. Make sure each NIC has flashed the latest version of NVM/firmware, > +if there > is. > + > +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 > +slots > can't provide enough bandwidth for 2x10G and above. Pci gen2 x 8 may not provide enough, pci gen2 x 16 may support that. > + > +4. When inserting NICs to the PCI slots, be careful about the NUMA. > +If you will + > +Example > +--- > + > +Below is an case of running dpdk l3fwd sample to get high performance > +with > Intel platform and NIC. > + > +**Note**: The scenario is to get best performance with two > +Intel(r)XL710 40G > ports. See below Figure1 as the performance test setup. > + > +.. figure:: img/pf_performance_test_setup.* > + > +**Figure 1. PF_Performance_Test_setup** > + > + > +1. Insert two NICs(Intel(r)XL710) into the platform, and use one port > +per card to > get best performance. The reason using two NICs is the PCIe Gen3's > limitations. > **Note**: As PCIe Gen3 can't provide 80G bandwidth for two 40G ports, > but two Pci gen3 x 8 cannot provide
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Thomas Monjalon > Sent: Monday, August 10, 2015 6:38 AM > To: Xu, Qian Q > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how > to get DPDK high perf on Intel platform. > > Hi, > > Thanks for adding more helpful documentation. > > 2015-08-10 14:34, Qian Xu: > > Signed-off-by: Qian Xu > > The Sign-off line must be below the commit message. > > > Add a new guide doc under guides folder. This document is a > > step-by-step guide about how to get high performance with DPDK on Intel's > platform and NICs. > > It is designed for users who are not familiar with DPDK but would like > > to measure the best performance. It contains step-by-step instructions to > > set > the platform and NICs to its best performance. > > The document will add more sections with the DPDK features' increment. > > Currently, the document has only one section about PF performance test > > setup, > and will add below cases in near future. > > > > * VF performance tuning. > > * Vhost/virtio performance tuning. > > * new features > > Good idea, thanks > > > --- /dev/null > > +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg > > + > + width="609.375" > > + height="349.6875" > > + preserveAspectRatio="none" > > + xlink:href="data:image/png;base64, > > This is a PNG picture embedded in SVG. > Please draw only some true SVG files. > Inkscape is the preferred tool. > > > +++ b/doc/guides/perf_tuning_guide/performance_tuning.rst > > This file is Intel-specific. So it should be renamed > doc/guides/perf_tuning/intel_platform.rst Agree with this. > > > +This document is a step-by-step guide for getting high DPDK performance > with Intel's platform and NICs. > > Could it be specific to Intel's CPU but allows some chapters for other NICs > tuning > (e.g. Chelsio, Cisco, Mellanox)? > Ideally, the NIC tuning should refer to sections in doc/guides/nics/. Yes, we want to have it as common as possible, and take Intel(r) NIC/Server as an example. > > > +1. Get an high end Intel? NIC, e.g: Intel? XL710. > > "Get an high end NIC that can sustain a high packet rate."? Usually yes, talking about 1G is meaningless now, and 10G @ line rate is not so challenging and complicated. 40G is more challenging, and complicated, so take it as an example. > > > +2. Make sure each NIC has flashed the latest version of NVM/firmware, if > there is. > > + > > +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots > can't provide enough bandwidth for 2x10G and above. > > Could you describe how to get this characteristics with lspci? > > > +4. When inserting NICs to the PCI slots, be careful about the NUMA. If you > > will > use 2 or more ports from different NICs, please make sure these NICs on the > same CPU socket. > > Could you describe how to know the CPU-PCI mapping? lstopo, lspci? Below there already have the descriptions. > > > +BIOS settings: > > +~~ > > + > > +1. To be sure, reset all the BIOS settings to default. > > + > > +2. Disable all power saving options, and set all options for best > > performance. > > + > > +3. Disable Turbo to ensure the performance scaling with core numbers > increment. > > + > > +4. Set memory frequency to the highest number, NOT auto. > > + > > +5. Disable all Virtualization options when you test physical function of > > NIC, and > turn on VT-d if you wants to use VFIO. > > Could you describe usage of dmidecode to check this, please? > > > +1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 > > - 64 > bits. > > Why not describe build options and run-time options to use with any Linux? Actually there is nothing special for performance testing, comparing to running DPDK basically. No need to duplicate here, as all the setup requirements are described in GSG. > > > +3. Make sure the widely used and fully validated version of kernel > > installed, > e.g. 3.18. > > Reference to release notes required. Agree with this. > > > +4. Make sure the required components are enabled for some old version of > kernels, before rebuilding the kernel. The kernel may need to be rebuilt if > any > DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org > for more details. > > Please use sphinx reference. > > > +1. For Intel? 40G NICs, special configurations should be set before > > compiling it, > as follows. **Note**: This is very important:: > > + > > + for at least DPDK release 1.8, 2.0 and 2.1, in > /config/common_linuxapp > > + CONFIG_RTE_PCI_CONFIG=y > > + CONFIG_RTE_PCI_EXTENDED_TAG=?on? > > Please insert it in a i40e doc instead of here. Then you can reference it. Good idea! Regards, Helin
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Qian Xu > Sent: Sunday, August 9, 2015 11:35 PM > To: dev at dpdk.org > Subject: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to > get DPDK high perf on Intel platform. > > Signed-off-by: Qian Xu > > Add a new guide doc under guides folder. This document is a step-by-step guide > about how to get high performance with DPDK on Intel's platform and NICs. > It is designed for users who are not familiar with DPDK but would like to > measure > the best performance. It contains step-by-step instructions to set the > platform > and NICs to its best performance. > The document will add more sections with the DPDK features' increment. > Currently, the document has only one section about PF performance test setup, > and will add below cases in near future. > > * VF performance tuning. > * Vhost/virtio performance tuning. > * new features > > > diff --git a/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg > b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg > new file mode 100644 > index 000..50ce92d > --- /dev/null > +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg > @@ -0,0 +1,375 @@ > + > + > + > + + xmlns:dc="http://purl.org/dc/elements/1.1/"; > + xmlns:cc="http://creativecommons.org/ns#"; > + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; > + xmlns:svg="http://www.w3.org/2000/svg"; > + xmlns="http://www.w3.org/2000/svg"; > + xmlns:xlink="http://www.w3.org/1999/xlink"; > + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"; > + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"; > + id="svg2" > + version="1.1" > + inkscape:version="0.91 r13725" > + width="609.375" > + height="349.6875" > + viewBox="0 0 609.375 349.6875" > + sodipodi:docname="Performance_test_setup.svg"> > + + id="metadata8"> > + > + + rdf:about=""> > +image/svg+xml > + + rdf:resource="http://purl.org/dc/dcmitype/StillImage"; /> > + > + > + > + > + + id="defs6" /> > + + pagecolor="#ff" > + bordercolor="#66" > + borderopacity="1" > + objecttolerance="10" > + gridtolerance="10" > + guidetolerance="10" > + inkscape:pageopacity="0" > + inkscape:pageshadow="2" > + inkscape:window-width="1596" > + inkscape:window-height="1108" > + id="namedview4" > + showgrid="false" > + inkscape:zoom="0.56123077" > + inkscape:cx="308.2511" > + inkscape:cy="174.84375" > + inkscape:window-x="0" > + inkscape:window-y="27" > + inkscape:window-maximized="1" > + inkscape:current-layer="svg2" /> > + + width="609.375" > + height="349.6875" > + preserveAspectRatio="none" > + > +xlink:href="data:image/png;base64,iVBORw0KGgoNSUhEUgAAAooAAAF1 > CAIAA > +ADgOlyKAXNSR0IArs4c6QRnQU1BAACx > +jwv8YQUJcEhZcwAADsMAAA7DAcdvqGQAAEUsSURBVHhe7d0JcBT3nS/w > v3yA1htqkcx > +l78qW > +0AgjLHadxY84owBCCIOgEstOVlbWSSG24hF6rofGu6YCu7AyDxJIOVmPXGULydlF > VHZjWS+xSVJI > +HDKHeYy9xGych2BsNDqCtmxhsERCDvARvf/VPd093aM5ejQ9o++npobunp6r0cx > 3ft3//v+ > +zxsbG > +CDjJTfJfcAzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAA > A4DuIZAADAcRD > +P > +joN4BgAAcBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZA > ADAcRDPjoN > +4BgAA > +cBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZAADAcRDPA > AAAjoN4BgAAcBz > +EMwAA > +gOMgngEAABwH8QwAAOA4WWNjY3ISIEWysrLkFABAhoo1bVE9AwAAOA7iGQA > AwHGwcxtST92 > +5jb9G > +AMgwcX+/oXoGAABwHMQzAACA4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMA > ADgO4hkAAMBxEM8 > +AMgwcX+AAACO > +g3gGAABwHMQzAACA4yCeAQAAHCfBPrevNHme9/bJGaaiemwLqVtxYkHbhoY8 > uSxuwfY9RS8 > +Snx0P > +BU6GPrcBIFOlss9tz47GsdeUy5ZiuTRBQydLV2wvGpjlkfMAAACTyATs3A7UrdieJS6 > 7AmJ > +R167t > +pe1XxLQGrcW31/n5xI6jJTT1DWHvfyVrxZ6mITkHAACQqZIdzzRxO3oef4LX1k/4Bj > pEKlc > +uW+gf > +uKysoMb25fN9C6vcdGJGQ2tjC5vQcz8y9hp2dAMAQOazIZ5btynF8YpXuuQyhf91 > b9/CrTU > +z+MyM > +htqF/hdfZ+vkzXR3B9jEUKCjj5CBK0E67Q+0VhRXsjUBAAAmNXuPPT9iEq6FM4vkF > E9lOVF > +cXfjB > +hSESPHWOPF7tI+cO0OmLH7gLZorbYXLKguSQ2xcA0kfyjz33Xe6VU7RWvs
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
Hi, Thanks for adding more helpful documentation. 2015-08-10 14:34, Qian Xu: > Signed-off-by: Qian Xu The Sign-off line must be below the commit message. > Add a new guide doc under guides folder. This document is a step-by-step > guide about how to get high performance with > DPDK on Intel's platform and NICs. > It is designed for users who are not familiar with DPDK but would like to > measure the best performance. It contains > step-by-step instructions to set the platform and NICs to its best > performance. > The document will add more sections with the DPDK features' increment. > Currently, the document has only one section about PF performance test setup, > and will add below cases in near future. > > * VF performance tuning. > * Vhost/virtio performance tuning. > * new features Good idea, thanks > --- /dev/null > +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg > + + width="609.375" > + height="349.6875" > + preserveAspectRatio="none" > + xlink:href="data:image/png;base64, This is a PNG picture embedded in SVG. Please draw only some true SVG files. Inkscape is the preferred tool. > +++ b/doc/guides/perf_tuning_guide/performance_tuning.rst This file is Intel-specific. So it should be renamed doc/guides/perf_tuning/intel_platform.rst > +This document is a step-by-step guide for getting high DPDK performance with > Intel's platform and NICs. Could it be specific to Intel's CPU but allows some chapters for other NICs tuning (e.g. Chelsio, Cisco, Mellanox)? Ideally, the NIC tuning should refer to sections in doc/guides/nics/. > +1. Get an high end Intel? NIC, e.g: Intel? XL710. "Get an high end NIC that can sustain a high packet rate."? > +2. Make sure each NIC has flashed the latest version of NVM/firmware, if > there is. > + > +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots > can't provide enough bandwidth for 2x10G and above. Could you describe how to get this characteristics with lspci? > +4. When inserting NICs to the PCI slots, be careful about the NUMA. If you > will use 2 or more ports from different NICs, please make sure these NICs on > the same CPU socket. Could you describe how to know the CPU-PCI mapping? lstopo, lspci? > +BIOS settings: > +~~ > + > +1. To be sure, reset all the BIOS settings to default. > + > +2. Disable all power saving options, and set all options for best > performance. > + > +3. Disable Turbo to ensure the performance scaling with core numbers > increment. > + > +4. Set memory frequency to the highest number, NOT auto. > + > +5. Disable all Virtualization options when you test physical function of > NIC, and turn on VT-d if you wants to use VFIO. Could you describe usage of dmidecode to check this, please? > +1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 - > 64 bits. Why not describe build options and run-time options to use with any Linux? > +3. Make sure the widely used and fully validated version of kernel > installed, e.g. 3.18. Reference to release notes required. > +4. Make sure the required components are enabled for some old version of > kernels, before rebuilding the kernel. The kernel may need to be rebuilt if > any DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org > for more details. Please use sphinx reference. > +1. For Intel? 40G NICs, special configurations should be set before > compiling it, as follows. **Note**: This is very important:: > + > + for at least DPDK release 1.8, 2.0 and 2.1, in > /config/common_linuxapp > + CONFIG_RTE_PCI_CONFIG=y > + CONFIG_RTE_PCI_EXTENDED_TAG=?on? Please insert it in a i40e doc instead of here. Then you can reference it.
[dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
Signed-off-by: Qian Xu Add a new guide doc under guides folder. This document is a step-by-step guide about how to get high performance with DPDK on Intel's platform and NICs. It is designed for users who are not familiar with DPDK but would like to measure the best performance. It contains step-by-step instructions to set the platform and NICs to its best performance. The document will add more sections with the DPDK features' increment. Currently, the document has only one section about PF performance test setup, and will add below cases in near future. * VF performance tuning. * Vhost/virtio performance tuning. * new features diff --git a/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg new file mode 100644 index 000..50ce92d --- /dev/null +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg @@ -0,0 +1,375 @@ + + + +http://purl.org/dc/elements/1.1/"; + xmlns:cc="http://creativecommons.org/ns#"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:svg="http://www.w3.org/2000/svg"; + xmlns="http://www.w3.org/2000/svg"; + xmlns:xlink="http://www.w3.org/1999/xlink"; + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"; + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"; + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + width="609.375" + height="349.6875" + viewBox="0 0 609.375 349.6875" + sodipodi:docname="Performance_test_setup.svg"> + + + +image/svg+xml +http://purl.org/dc/dcmitype/StillImage"; /> + + + + + + + + diff --git a/doc/guides/perf_tuning_guide/index.rst b/doc/guides/perf_tuning_guide/index.rst new file mode 100644 index 000..ff325e9 --- /dev/null +++ b/doc/guides/perf_tuning_guide/index.rst @@ -0,0 +1,47 @@ +.. BSD LICENSE +Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +* Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in +the documentation and/or other materials provided with the +distribution. +* Neither the name of Intel Corporation nor the names of its +contributors may be used to endorse or promote products derived +from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +Performance Tuning Guide for Intel's platform += + +|today| + +Contents + +.. toctree:: +:maxdepth: 2 +:numbered: + +intro +performance_tuning + + + + diff --git a/doc/guides/perf_tuning_guide/intro.rst b/doc/guides/perf_tuning_guide/intro.rst new file mode 100644 index 000..5672549 --- /dev/null +++ b/doc/guides/perf_tuning_guide/intro.rst @@ -0,0 +1,44 @@ +.. BSD LICENSE +Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +* Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in +the documentation and/or other materials provided with the +distribution. +* Neither the name of Intel Corporation nor the names of its +contributors may be used to endorse or promote products derived +from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, T
