[dpdk-dev] DPDK Port Mirroring
Now; what if we are to encapsulate the packet within a tunnel; such as GRE, prior to sending it out to a mirrored-port ... Must the packet be copied into a new buffer or can we use a DPDK primitive that is more efficient? Sami. > -Original Message- > From: Bruce Richardson [mailto:bruce.richardson at intel.com] > Sent: Friday, July 10, 2015 9:48 AM > To: Sanford, Robert > Cc: Matt Laswell; Wiles, Keith; dev at dpdk.org; Assaad, Sami (Sami) > Subject: Re: [dpdk-dev] DPDK Port Mirroring > On Fri, Jul 10, 2015 at 01:46:21PM +, Sanford, Robert wrote: > > Silly questions: Why use rte_pktmbuf_clone()? Assuming that one is not > > going to modify the mbuf at all, why not just increment the reference > > count with rte_mbuf_refcnt_update()? > > > Yep, that should work fine too. > > /Bruce > > -- > > Thanks, > > Robert > > > > > > >Keith speaks truth. If I were going to do what you're describing, I > > >would do the following: > > > > > >1. Start with the l2fwd example application. > > >2. Remove the part where it modifies the ethernet MAC address of > > >received packets. > > >3. Add a call in to clone mbufs via rte_pktmbuf_clone() and send the > > >cloned packets out of the port of your choice > > > > > >As long as you don't need to modify the packets - and if you're > > >mirroring, you shouldn't - simply cloning received packets and > > >sending them out your mirror port should get you most of the way there. > > > > > >On Thu, Jul 9, 2015 at 3:17 PM, Wiles, Keith > > >wrote: > > > > > >> > > >> > > >> On 7/9/15, 12:26 PM, "dev on behalf of Assaad, Sami (Sami)" > > >> > > >>wrote: > > >> > > >> >Hello, > > >> > > > >> >I want to build a DPDK app that is able to port-mirror all ingress > > >> >traffic from two 10G interfaces. > > >> > > > >> >1. Is it possible in port-mirroring traffic consisting of 450byte > > >> >packets at 20G without losing more than 5% of traffic? > > >> > > > >> >2. Would you have any performance results due to packet copying? > > >> > > >> Do you need to copy the packet if you increment the reference count > > >> you can send the packet to both ports without having to copy the packet. > > >> > > > >> >3. Would you have any port mirroring DPDK sample code? > > >> > > >> DPDK does not have port mirroring example, but you could grab the > > >>l2fwd or l3fwd and modify it to do what you want. > > >> > > > >> >Thanks in advance. > > >> > > > >> >Best Regards, > > >> >Sami Assaad. > > >> > > >> > >
[dpdk-dev] DPDK Port Mirroring
Big Thanks! Sami. From: Matt Laswell [mailto:lasw...@infiniteio.com] Sent: Thursday, July 09, 2015 4:28 PM To: Wiles, Keith Cc: Assaad, Sami (Sami); dev at dpdk.org; Richardson, Bruce Subject: Re: [dpdk-dev] DPDK Port Mirroring Keith speaks truth. If I were going to do what you're describing, I would do the following: 1. Start with the l2fwd example application. 2. Remove the part where it modifies the ethernet MAC address of received packets. 3. Add a call in to clone mbufs via rte_pktmbuf_clone() and send the cloned packets out of the port of your choice As long as you don't need to modify the packets - and if you're mirroring, you shouldn't - simply cloning received packets and sending them out your mirror port should get you most of the way there. On Thu, Jul 9, 2015 at 3:17 PM, Wiles, Keith mailto:keith.wiles at intel.com>> wrote: On 7/9/15, 12:26 PM, "dev on behalf of Assaad, Sami (Sami)" mailto:dev-bounces at dpdk.org> on behalf of sami.assaad at alcatel-lucent.com<mailto:sami.assaad at alcatel-lucent.com>> wrote: >Hello, > >I want to build a DPDK app that is able to port-mirror all ingress >traffic from two 10G interfaces. > >1. Is it possible in port-mirroring traffic consisting of 450byte >packets at 20G without losing more than 5% of traffic? > >2. Would you have any performance results due to packet copying? Do you need to copy the packet if you increment the reference count you can send the packet to both ports without having to copy the packet. > >3. Would you have any port mirroring DPDK sample code? DPDK does not have port mirroring example, but you could grab the l2fwd or l3fwd and modify it to do what you want. > >Thanks in advance. > >Best Regards, >Sami Assaad.
[dpdk-dev] dev Digest, Vol 45, Issue 9
Thank you! Best Regards, Sami. -Original Message- From: Bruce Richardson [mailto:bruce.richard...@intel.com] Sent: Thursday, July 09, 2015 6:10 AM To: Assaad, Sami (Sami) Cc: dev at dpdk.org Subject: Re: dev Digest, Vol 45, Issue 9 On Wed, Jul 08, 2015 at 11:18:04PM +, Assaad, Sami (Sami) wrote: > Hello Bruce, > > As you stated previously, I cannot guarantee the proper mapping of the > virtual addresses between my primary and secondary processes. I have > one primary process and up to 42 secondary processes. The application > as a whole works very well. However, closing the > application/restarting it continuously results with certain clients > (secondary processes) failing to memory map the primary virtual > addresses into /dev/zero (fd_zero). [Error "Cound not mmap 8573157376 > bytes in /dev/zero to requested address ..."] > > So, I disabled ASLR on my virtual machine (VM). Rebooted it with ASLR > permanently disabled and ran the application. None of the clients (secondary > processes) was able in memory mapping the virtual addresses derived from the > primary process. Rte_eal_config_reattach() fails with the error "Cannot mmap > memory for rte_config". > > My experience with ASLR is limited. Must I rebuild the software with ASLR > disabled or am I missing EAL configuration? > How about SELinux - must it be enabled or disabled? > Or ... must I manually configure the virtual addressing for both the primary > & secondary processes? An example would help. > > I am running the primary process with the option --proc-type=primary and the > 40+ clients with --proc-type=secondary. > > - My OS is CentOS6.6 > - Using KVM/QEMU > - DPDK 1.8.0 > - Haswell > > Thanks in advance! > > Best Regards, > Sami Assaad. > This unfortunately can happen. To work around this we added in the EAL flag "--base-virtaddr", to allow you to hint a suitable base address in the primary process so that the address mappings can succeed in the secondary processes. [What is happening with ASLR off is that the mappings are repeatable - either always successful or always fail - sadly you hit the latter]. One tip might be to look at the output of the secondary processes to see what address the failing file is getting mapped at, and use that as a base address hint to the primary process. You may have to play about with a few values before you get it working. [One other possibility that sometimes works is to pass in additional cores in the coremask to the primary process. Having more threads, even if they are idle, can also adjust the address layout due to the addition per-thread stack space.] /Bruce > > -Original Message- > > Message: 2 > Date: Mon, 15 Jun 2015 11:21:39 +0100 > From: Bruce Richardson > To: "Assaad, Sami (Sami)" > Cc: "dev at dpdk.org" > Subject: Re: [dpdk-dev] DPDK and ASLR > Message-ID: <20150615102138.GB3872 at bricha3-MOBL3> > Content-Type: text/plain; charset=us-ascii > > On Fri, Jun 12, 2015 at 10:53:58PM +, Assaad, Sami (Sami) wrote: > > When I operate a DPDK based application, the EAL always reports the > > following: > > EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the > > kernel. > > EAL: This may cause issues with mapping memory into secondary > > processes. > > > > Our application is DPDK client/server based and runs properly. > > > > My questions are: > > > > * Is this warning of any importance? > > Yes, it's there for a reason. With ASLR, the position of the hugepage (and > other) memory in your DPDK primary process virtual address space will move > about from one run to another, and the same with the secondary process. > Because of this, you may occasionally get instances where your application > fails to run because an essential piece of memory is mapped at address X in > the primary, while something else is mapped at address X in the secondary > process. How frequently, if ever, this happens will vary from application to > application. > > If ASLR is disabled, the memory mappings created in the primary and secondary > processes will be identical and repeatable from one run to another, so you > can know that if a set of processes starts once, it will start a second time. > With ASLR enabled, that guarantee cannot be made. > > > > > > * Should ASLR be disabled? > > > > That is a questions we can't answer for you. ASLR is a security feature in > the OS so you should be aware of the implications of disabling it. However, > if you need absolute guarantees of repeatabiltiy of mappings from one > multi-process run to another, the only way get that - that I am aware of - is > to disable ASLR. If an occasional random failure at startup is ok, then ASLR > can safely be left on. > > > * Does ASLR affect DPDK performance? > > No, it only affects the repeatability of memory mappings at DPDK start-up. > > Hope this clarifies things. > > /Bruce >
[dpdk-dev] DPDK Port Mirroring
Hello, I want to build a DPDK app that is able to port-mirror all ingress traffic from two 10G interfaces. 1. Is it possible in port-mirroring traffic consisting of 450byte packets at 20G without losing more than 5% of traffic? 2. Would you have any performance results due to packet copying? 3. Would you have any port mirroring DPDK sample code? Thanks in advance. Best Regards, Sami Assaad.
[dpdk-dev] dev Digest, Vol 45, Issue 9
Hello Bruce, As you stated previously, I cannot guarantee the proper mapping of the virtual addresses between my primary and secondary processes. I have one primary process and up to 42 secondary processes. The application as a whole works very well. However, closing the application/restarting it continuously results with certain clients (secondary processes) failing to memory map the primary virtual addresses into /dev/zero (fd_zero). [Error "Cound not mmap 8573157376 bytes in /dev/zero to requested address ..."] So, I disabled ASLR on my virtual machine (VM). Rebooted it with ASLR permanently disabled and ran the application. None of the clients (secondary processes) was able in memory mapping the virtual addresses derived from the primary process. Rte_eal_config_reattach() fails with the error "Cannot mmap memory for rte_config". My experience with ASLR is limited. Must I rebuild the software with ASLR disabled or am I missing EAL configuration? How about SELinux - must it be enabled or disabled? Or ... must I manually configure the virtual addressing for both the primary & secondary processes? An example would help. I am running the primary process with the option --proc-type=primary and the 40+ clients with --proc-type=secondary. - My OS is CentOS6.6 - Using KVM/QEMU - DPDK 1.8.0 - Haswell Thanks in advance! Best Regards, Sami Assaad. -Original Message- Message: 2 Date: Mon, 15 Jun 2015 11:21:39 +0100 From: Bruce Richardson To: "Assaad, Sami (Sami)" Cc: "dev at dpdk.org" Subject: Re: [dpdk-dev] DPDK and ASLR Message-ID: <20150615102138.GB3872 at bricha3-MOBL3> Content-Type: text/plain; charset=us-ascii On Fri, Jun 12, 2015 at 10:53:58PM +, Assaad, Sami (Sami) wrote: > When I operate a DPDK based application, the EAL always reports the following: > EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the > kernel. > EAL: This may cause issues with mapping memory into secondary processes. > > Our application is DPDK client/server based and runs properly. > > My questions are: > > * Is this warning of any importance? Yes, it's there for a reason. With ASLR, the position of the hugepage (and other) memory in your DPDK primary process virtual address space will move about from one run to another, and the same with the secondary process. Because of this, you may occasionally get instances where your application fails to run because an essential piece of memory is mapped at address X in the primary, while something else is mapped at address X in the secondary process. How frequently, if ever, this happens will vary from application to application. If ASLR is disabled, the memory mappings created in the primary and secondary processes will be identical and repeatable from one run to another, so you can know that if a set of processes starts once, it will start a second time. With ASLR enabled, that guarantee cannot be made. > > * Should ASLR be disabled? > That is a questions we can't answer for you. ASLR is a security feature in the OS so you should be aware of the implications of disabling it. However, if you need absolute guarantees of repeatabiltiy of mappings from one multi-process run to another, the only way get that - that I am aware of - is to disable ASLR. If an occasional random failure at startup is ok, then ASLR can safely be left on. > * Does ASLR affect DPDK performance? No, it only affects the repeatability of memory mappings at DPDK start-up. Hope this clarifies things. /Bruce
[dpdk-dev] 10G Interface used as PCI Pass-Through reports 64bytes / packet
Thanks Bruce for your response. This is a very odd issue. I'm connecting a Pktgen-DPDK server directly to our application server running a DPDK Server/Client Process architecture. I'm constantly seeing 64bytes per packet, no matter what packet size is being received by the NIC running on the application server. I initially thought that maybe the NIC was configured in a particular mode to improve the throughput through the SR-IOV links. Based on your response, this is a wrong assumption. Interesting...I have tried both a HP and Intel NIC, DPDK 1.8 and 2.0, and DPDK examples, all of which report 64 bytes/packet by the NIC ports. I'm using CentOS 6.6 (2.6.32-504.23.4.el6.x86_64). I'm wondering if my PCI pass-through is setup properly (?) ... yet, all the network traffic is being handled as expected by the designed DPDK application. In case the issue is with the VM configuration (which I seriously doubt), I've copied my VM XML here: vm-sami 1eda9ae3-0155-de14-6e1c-0fbe0aa880f6 10240 10240 46 hvm Haswell Intel destroy restart restart /usr/libexec/qemu-kvm Best Regards, Sami Assaad. -Original Message- From: Bruce Richardson [mailto:bruce.richard...@intel.com] Sent: Monday, June 29, 2015 5:19 AM To: Assaad, Sami (Sami) Cc: dev at dpdk.org Subject: Re: [dpdk-dev] 10G Interface used as PCI Pass-Through reports 64bytes / packet On Fri, Jun 26, 2015 at 04:56:18PM +, Assaad, Sami (Sami) wrote: > Hello, > > Is it normal that a 10G NIC interface, supporting the 82599 Ethernet > Controller, configured as PCI Pass-through for a virtual machine using DPDK, > reports 64 bytes per packet; no matter what the packet size? > That would not be expected behaviour, no. AFAIK, the 82599 NIC counters should behave in the same way whether or not it is passed through to a VM or used on a host. > If so; I'm assuming this is to improve the performance of passing the network > traffic to the VM. Is there a way to configure the NIC to properly present > the proper byte count/packet? > I'm not sure what you mean here. I can't see how the reporting of byte-counts would affect performance. Can you clarify what exactly you are seeing, and why you think there is a performance benefit because of it? /Bruce > Thanks in advance. > > Best Regards, > Sami.
[dpdk-dev] 10G Interface used as PCI Pass-Through reports 64bytes / packet
Hello, Is it normal that a 10G NIC interface, supporting the 82599 Ethernet Controller, configured as PCI Pass-through for a virtual machine using DPDK, reports 64 bytes per packet; no matter what the packet size? If so; I'm assuming this is to improve the performance of passing the network traffic to the VM. Is there a way to configure the NIC to properly present the proper byte count/packet? Thanks in advance. Best Regards, Sami.
[dpdk-dev] DPDK-Pktgen
Hello. I am using DPDK-Pktgen (Pktgen version 2.77 using DPDK-1.7.1). My NIC consists of two 10G ports. Now, my question is : How are the port MBits/s Rx/Tx calculated? Example: I am running the traffic generator with the following configuration: * 400 Byte Packet Sizes * At Line Rate (i.e: 100%) * Unidirectional (Tx only) The Pkts/s TX is 2985600. (Looks correct) But the MBits/s Tx is set to 2006 (per port Tx). How does DPDK-Pktgen calculate 2006 MBits/s Tx ? Is this a bug? Thanks in advance. Best Regards, Sami Assaad.
[dpdk-dev] DPDK and ASLR
When I operate a DPDK based application, the EAL always reports the following: EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel. EAL: This may cause issues with mapping memory into secondary processes. Our application is DPDK client/server based and runs properly. My questions are: * Is this warning of any importance? * Should ASLR be disabled? * Does ASLR affect DPDK performance? _ Issue Update: 7. Re: How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)? (Qiu, Michael) The problem I had was due to the fact that RMRR was used by the HOST for the HP 10G NIC card; which prevented me from performing a PCI Pass-Through. HP provided a BIOS and NIC Firmware update to resolve this. It is to note that an Intel NIC having an 82599 Ethernet Controller did not generate the RMRR issue! Best Regards, Sami Assaad.
[dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
#x27;org.freedesktop.nm_dispatcher' May 19 15:10:12 ni-nfvhost01 systemd: Started Network Manager Script Dispatcher Service. May 19 15:10:12 ni-nfvhost01 nm-dispatcher: Dispatching action 'up' for vnet0 May 19 15:10:12 ni-nfvhost01 kvm: 1 guest now active May 19 15:10:12 ni-nfvhost01 systemd: Unit iscsi.service cannot be reloaded because it is inactive. May 19 15:10:12 ni-nfvhost01 kernel: vfio-pci :04:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor. May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered disabled state May 19 15:10:12 ni-nfvhost01 kernel: device vnet0 left promiscuous mode May 19 15:10:12 ni-nfvhost01 kernel: virbr0: port 2(vnet0) entered disabled state May 19 15:10:12 ni-nfvhost01 avahi-daemon[1280]: Withdrawing workstation service for vnet0. May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: (vnet0): device state change: activated -> unmanaged (reason 'removed') [100 10 36] May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: (vnet0): deactivating device (reason 'removed') [36] May 19 15:10:12 ni-nfvhost01 NetworkManager[1371]: (virbr0): failed to detach bridge port vnet0 May 19 15:10:12 ni-nfvhost01 nm-dispatcher: Dispatching action 'down' for vnet0 May 19 15:10:12 ni-nfvhost01 journal: Unable to read from monitor: Connection reset by peer May 19 15:10:12 ni-nfvhost01 journal: internal error: early end of file from monitor: possible problem: 2015-05-19T19:10:12.674077Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to set iommu for container: Operation not permitted 2015-05-19T19:10:12.674118Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to setup container for group 19 2015-05-19T19:10:12.674128Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to get group 19 2015-05-19T19:10:12.674141Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: Device initialization failed. 2015-05-19T19:10:12.674155Z qemu-kvm: -device vfio-pci,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x9: Device 'vfio-pci' could not be initialized May 19 15:10:12 ni-nfvhost01 kvm: 0 guests now active May 19 15:10:12 ni-nfvhost01 systemd-machined: Machine qemu-vNIDS-VM1 terminated. May 19 15:11:01 ni-nfvhost01 systemd: Created slice user-0.slice. May 19 15:11:01 ni-nfvhost01 systemd: Starting Session 329 of user root. Overall Hypothesis: The issue seems to be related to the Ethernet Controller's interfaces which I'm trying to bring into the VM. My Ethernet Controller is : Intel 10G x540-AT2 (rev 01). The problem is associated to RMRR. Can this issue be attributed to my BIOS? My Bios is the following: ProLiant System BIOS P89 V1.21 11/03/2014. Thanks in advance. Best Regards, Sami. -Original Message- From: Qiu, Michael [mailto:michael@intel.com] Sent: Monday, May 18, 2015 6:01 AM To: Assaad, Sami (Sami); Richardson, Bruce Cc: dev at dpdk.org Subject: Re: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)? Hi, Sami Could you mind to supply the syslog? Especially iommu related parts. Also you could update the qemu or kernel to see if this issue still exists. Thanks, Michael On 5/16/2015 3:31 AM, Assaad, Sami (Sami) wrote: > On Fri, May 15, 2015 at 12:54:19PM +, Assaad, Sami (Sami) wrote: >> Thanks Bruce for your reply. >> >> Yes, your idea of bringing the PF into the VM looks like an option. However, >> how do you configure the physical interfaces within the VM supporting SRIOV? >> I always believed that the VM needed to be associated with a >> virtual/emulated interface card. With your suggestion, I would actually >> configure the physical interface card/non-emulated within the VM. >> >> If you could provide me some example configuration commands, it would be >> really appreciated. >> > You'd pass in the PF in the same way as the VF, just skip all the steps > creating the VF on the host. To the system and hypervisor, both are just PCI > devices! > > As for configuration, the setup and configuration of the PF in the guest is > exactly the same as on the host - it's the same hardware with the same PCI > bars. > It's the IOMMU on your platform that takes care of memory isolation and > address translation and that should work with either PF or VF. > > Regards, > /Bruce > >> Thanks in advance. >> >> Best Regards, >> Sami. >> >> -----Original Message- >> From: Bruce Richardson [mailto:bruce.richardson at intel.com] >> Sent: Friday, May 15, 2015 5:27 AM >> To: Stephen Hemminger >> Cc: Assaad, Sami (Sami); dev at dpdk.org >> S
[dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
On Fri, May 15, 2015 at 12:54:19PM +, Assaad, Sami (Sami) wrote: > Thanks Bruce for your reply. > > Yes, your idea of bringing the PF into the VM looks like an option. However, > how do you configure the physical interfaces within the VM supporting SRIOV? > I always believed that the VM needed to be associated with a virtual/emulated > interface card. With your suggestion, I would actually configure the physical > interface card/non-emulated within the VM. > > If you could provide me some example configuration commands, it would be > really appreciated. > You'd pass in the PF in the same way as the VF, just skip all the steps creating the VF on the host. To the system and hypervisor, both are just PCI devices! As for configuration, the setup and configuration of the PF in the guest is exactly the same as on the host - it's the same hardware with the same PCI bars. It's the IOMMU on your platform that takes care of memory isolation and address translation and that should work with either PF or VF. Regards, /Bruce > Thanks in advance. > > Best Regards, > Sami. > > -Original Message- > From: Bruce Richardson [mailto:bruce.richardson at intel.com] > Sent: Friday, May 15, 2015 5:27 AM > To: Stephen Hemminger > Cc: Assaad, Sami (Sami); dev at dpdk.org > Subject: Re: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI > Pass-Through (SR-IOV)? > > On Thu, May 14, 2015 at 04:47:19PM -0700, Stephen Hemminger wrote: > > On Thu, 14 May 2015 21:38:24 + > > "Assaad, Sami (Sami)" wrote: > > > > > Hello, > > > > > > My Hardware consists of the following: > > > - DL380 Gen 9 Server supporting two Haswell Processors (Xeon CPU > > > E5-2680 v3 @ 2.50GHz) > > > - An x540 Ethernet Controller Card supporting 2x10G ports. > > > > > > Software: > > > - CentOS 7 (3.10.0-229.1.2.el7.x86_64) > > > - DPDK 1.8 > > > > > > I want all the network traffic received on the two 10G ports to be > > > transmitted to my VM. The issue is that the Virtual Function / Physical > > > Functions have setup the internal virtual switch to only route Ethernet > > > packets with destination MAC address matching the VM virtual interface > > > MAC. How can I configure my virtual environment to provide all network > > > traffic to the VM...i.e. set the virtual functions for both PCI devices > > > in Promiscuous mode? > > > > > > [ If a l2fwd-vf example exists, this would actually solve this > > > problem ... Is there a DPDK l2fwd-vf example available? ] > > > > > > > > > Thanks in advance. > > > > > > Best Regards, > > > Sami Assaad. > > > > This is a host side (not DPDK) issue. > > > > Intel PF driver will not allow guest (VF) to go into promiscious > > mode since it would allow traffic stealing which is a security violation. > > Could you maybe try passing the PF directly into the VM, rather than a VF > based off it? Since you seem to want all traffic to go to the one VM, there > seems little point in creating a VF on the device, and should let the VM > control the whole NIC directly. > > Regards, > /Bruce Hi Bruce, I was provided two options: 1. Pass the PF directly into the VM 2. Use ixgbe VF mirroring I decided to first try your proposal of passing the PF directly into the VM. However, I ran into some issues. But prior to providing the problem details, the following is my server environment: I'm using CentOS 7 KVM/QEMU [root at ni-nfvhost01 qemu]# uname -a Linux ni-nfvhost01 3.10.0-229.1.2.el7.x86_64 #1 SMP Fri Mar 27 03:04:26 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux [root at ni-nfvhost01 qemu]# lspci -n -s 04:00.0 04:00.0 0200: 8086:1528 (rev 01) [root at ni-nfvhost01 qemu]# lspci | grep -i eth 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 02:00.2 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 02:00.3 Ethernet controller: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) 04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) - The following is my grub execution: [root at ni-nfvhost01 qemu]# cat /proc/cmdline BOOT_IMAGE=/vmlinuz-3.10.0-229.1.2.el7.x86_64 root=/dev/mapper/centos-root ro rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun17 rd.lvm.lv=centos/root crashkernel=auto vconsole.keymap=
[dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
Thanks Konstantin for your reply. Konstantin, Would you be able in providing more details as to ixgbe VF mirroring? Would you have any valuable URL's or documentation that explain this in detail and/or provide examples? Thanks in advance. Best Regards, Sami. -Original Message- From: Ananyev, Konstantin [mailto:konstantin.anan...@intel.com] Sent: Friday, May 15, 2015 5:31 AM To: Richardson, Bruce; Stephen Hemminger Cc: dev at dpdk.org; Assaad, Sami (Sami) Subject: RE: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)? > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Bruce Richardson > Sent: Friday, May 15, 2015 10:27 AM > To: Stephen Hemminger > Cc: dev at dpdk.org; Assaad, Sami (Sami) > Subject: Re: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI > Pass-Through (SR-IOV)? > > On Thu, May 14, 2015 at 04:47:19PM -0700, Stephen Hemminger wrote: > > On Thu, 14 May 2015 21:38:24 + > > "Assaad, Sami (Sami)" wrote: > > > > > Hello, > > > > > > My Hardware consists of the following: > > > - DL380 Gen 9 Server supporting two Haswell Processors (Xeon CPU > > > E5-2680 v3 @ 2.50GHz) > > > - An x540 Ethernet Controller Card supporting 2x10G ports. > > > > > > Software: > > > - CentOS 7 (3.10.0-229.1.2.el7.x86_64) > > > - DPDK 1.8 > > > > > > I want all the network traffic received on the two 10G ports to be > > > transmitted to my VM. The issue is that the Virtual Function / > Physical Functions have setup the internal virtual switch to only > route Ethernet packets with destination MAC address matching the VM > virtual interface MAC. How can I configure my virtual environment to provide > all network traffic to the VM...i.e. set the virtual functions for both PCI > devices in Promiscuous mode? > > > > > > [ If a l2fwd-vf example exists, this would actually solve this > > > problem ... Is there a DPDK l2fwd-vf example available? ] > > > > > > > > > Thanks in advance. > > > > > > Best Regards, > > > Sami Assaad. > > > > This is a host side (not DPDK) issue. > > > > Intel PF driver will not allow guest (VF) to go into promiscious > > mode since it would allow traffic stealing which is a security violation. > > Could you maybe try passing the PF directly into the VM, rather than a > VF based off it? Since you seem to want all traffic to go to the one > VM, there seems little point in creating a VF on the device, and > should let the VM control the whole NIC directly. As I remember, with ixgbe it is possible to make one VF a mirror (receive a copy of all traffic). Konstantin > > Regards, > /Bruce
[dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
Thanks Bruce for your reply. Yes, your idea of bringing the PF into the VM looks like an option. However, how do you configure the physical interfaces within the VM supporting SRIOV? I always believed that the VM needed to be associated with a virtual/emulated interface card. With your suggestion, I would actually configure the physical interface card/non-emulated within the VM. If you could provide me some example configuration commands, it would be really appreciated. Thanks in advance. Best Regards, Sami. -Original Message- From: Bruce Richardson [mailto:bruce.richard...@intel.com] Sent: Friday, May 15, 2015 5:27 AM To: Stephen Hemminger Cc: Assaad, Sami (Sami); dev at dpdk.org Subject: Re: [dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)? On Thu, May 14, 2015 at 04:47:19PM -0700, Stephen Hemminger wrote: > On Thu, 14 May 2015 21:38:24 + > "Assaad, Sami (Sami)" wrote: > > > Hello, > > > > My Hardware consists of the following: > > - DL380 Gen 9 Server supporting two Haswell Processors (Xeon CPU E5-2680 > > v3 @ 2.50GHz) > > - An x540 Ethernet Controller Card supporting 2x10G ports. > > > > Software: > > - CentOS 7 (3.10.0-229.1.2.el7.x86_64) > > - DPDK 1.8 > > > > I want all the network traffic received on the two 10G ports to be > > transmitted to my VM. The issue is that the Virtual Function / Physical > > Functions have setup the internal virtual switch to only route Ethernet > > packets with destination MAC address matching the VM virtual interface MAC. > > How can I configure my virtual environment to provide all network traffic > > to the VM...i.e. set the virtual functions for both PCI devices in > > Promiscuous mode? > > > > [ If a l2fwd-vf example exists, this would actually solve this > > problem ... Is there a DPDK l2fwd-vf example available? ] > > > > > > Thanks in advance. > > > > Best Regards, > > Sami Assaad. > > This is a host side (not DPDK) issue. > > Intel PF driver will not allow guest (VF) to go into promiscious mode > since it would allow traffic stealing which is a security violation. Could you maybe try passing the PF directly into the VM, rather than a VF based off it? Since you seem to want all traffic to go to the one VM, there seems little point in creating a VF on the device, and should let the VM control the whole NIC directly. Regards, /Bruce
[dpdk-dev] How do you setup a VM in Promiscuous Mode using PCI Pass-Through (SR-IOV)?
Hello, My Hardware consists of the following: - DL380 Gen 9 Server supporting two Haswell Processors (Xeon CPU E5-2680 v3 @ 2.50GHz) - An x540 Ethernet Controller Card supporting 2x10G ports. Software: - CentOS 7 (3.10.0-229.1.2.el7.x86_64) - DPDK 1.8 I want all the network traffic received on the two 10G ports to be transmitted to my VM. The issue is that the Virtual Function / Physical Functions have setup the internal virtual switch to only route Ethernet packets with destination MAC address matching the VM virtual interface MAC. How can I configure my virtual environment to provide all network traffic to the VM...i.e. set the virtual functions for both PCI devices in Promiscuous mode? [ If a l2fwd-vf example exists, this would actually solve this problem ... Is there a DPDK l2fwd-vf example available? ] Thanks in advance. Best Regards, Sami Assaad.
