[dpdk-dev] Error building using 1.6.0r1
On 02/28/2014 09:47 AM, Mats Liljegren wrote: > On Fri, Feb 28, 2014 at 9:11 AM, David Marchand > wrote: >> Hello Mats, >> >> I reproduced the problem (and another one). >> I will send two patches in a few minutes, can you try them ? >> >> >> Thank you. > > Yes, I can do that. > > Best regards > Mats Liljegren > I have now verified that using the following two patches: [PATCH 1/2] eal: fix use of RTE_PTR_ALIGN_CEIL macro on 32bits system [PATCH 2/2] mem: fix build on 32bits system fixed the build issues. I now have a new issue: I get the following error: EAL: Error - exiting with code: 1 Cause: Cannot init mbuf pool The code causing this error is ripped from l2fwd example. I did not get this error using DPDK 1.5.2. The question is whether this is an unrelated issue or not... This is the complete log for DPDK 1.6.2 + patches: Start snippet EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Skip lcore 4 (not detected) EAL: Skip lcore 5 (not detected) EAL: Skip lcore 6 (not detected) EAL: Skip lcore 7 (not detected) EAL: Skip lcore 8 (not detected) EAL: Skip lcore 9 (not detected) EAL: Skip lcore 10 (not detected) EAL: Skip lcore 11 (not detected) EAL: Skip lcore 12 (not detected) EAL: Skip lcore 13 (not detected) EAL: Skip lcore 14 (not detected) EAL: Skip lcore 15 (not detected) EAL: Skip lcore 16 (not detected) EAL: Skip lcore 17 (not detected) EAL: Skip lcore 18 (not detected) EAL: Skip lcore 19 (not detected) EAL: Skip lcore 20 (not detected) EAL: Skip lcore 21 (not detected) EAL: Skip lcore 22 (not detected) EAL: Skip lcore 23 (not detected) EAL: Skip lcore 24 (not detected) EAL: Skip lcore 25 (not detected) EAL: Skip lcore 26 (not detected) EAL: Skip lcore 27 (not detected) EAL: Skip lcore 28 (not detected) EAL: Skip lcore 29 (not detected) EAL: Skip lcore 30 (not detected) EAL: Skip lcore 31 (not detected) EAL: Skip lcore 32 (not detected) EAL: Skip lcore 33 (not detected) EAL: Skip lcore 34 (not detected) EAL: Skip lcore 35 (not detected) EAL: Skip lcore 36 (not detected) EAL: Skip lcore 37 (not detected) EAL: Skip lcore 38 (not detected) EAL: Skip lcore 39 (not detected) EAL: Skip lcore 40 (not detected) EAL: Skip lcore 41 (not detected) EAL: Skip lcore 42 (not detected) EAL: Skip lcore 43 (not detected) EAL: Skip lcore 44 (not detected) EAL: Skip lcore 45 (not detected) EAL: Skip lcore 46 (not detected) EAL: Skip lcore 47 (not detected) EAL: Skip lcore 48 (not detected) EAL: Skip lcore 49 (not detected) EAL: Skip lcore 50 (not detected) EAL: Skip lcore 51 (not detected) EAL: Skip lcore 52 (not detected) EAL: Skip lcore 53 (not detected) EAL: Skip lcore 54 (not detected) EAL: Skip lcore 55 (not detected) EAL: Skip lcore 56 (not detected) EAL: Skip lcore 57 (not detected) EAL: Skip lcore 58 (not detected) EAL: Skip lcore 59 (not detected) EAL: Skip lcore 60 (not detected) EAL: Skip lcore 61 (not detected) EAL: Skip lcore 62 (not detected) EAL: Skip lcore 63 (not detected) EAL: Auto-detected process type: PRIMARY EAL: Setting up memory... EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf7a0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf760 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf720 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf6e0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf6a0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf660 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf620 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf5e0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf5a0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf560 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf520 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf4e0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf4a0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf460 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf420 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf3e0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf3a0 (size = 0x20) EAL: Ask a virtual area of 0x20 bytes EAL: Virtual area found at 0xf360 (size = 0x20) EAL: Ask a vi
[dpdk-dev] Error building using 1.6.0r1
On Fri, Feb 28, 2014 at 9:11 AM, David Marchand wrote: > Hello Mats, > > I reproduced the problem (and another one). > I will send two patches in a few minutes, can you try them ? > > > Thank you. Yes, I can do that. Best regards Mats Liljegren
[dpdk-dev] Issues with multiple DPDK instances
I'm trying to get multiple DPDK instances (processes) but seems to have problems with getting the multiple queues that this (as far as I understand it) requires. I'm using the following command line for the host: sudo insmod igb max_vfs=0,1,1,0 VMDQ=0,2,2,0 The idea here is that first and last interface will be kept on host, while the two middle interfaces will be given to guest as virtual functions. When starting the primary, it fails saying "-22" in call to rte_eth_dev_configure(), which means EINVAL. I have noticed that if I set 1 queue for rx and tx, it starts. But setting 2 queues for rx and tx respectively, it fails. Am I using the wrong parameters to IGB? Best regards Mats Liljegren
[dpdk-dev] Error building using 1.6.0r1
I run a 64-bit host building for 32-bit DPDK, which fails with: dpdk/i686-default-linuxapp-gcc$ make == Build scripts == Build scripts/testhost == Build lib == Build lib/librte_eal == Build lib/librte_eal/common == Build lib/librte_eal/linuxapp == Build lib/librte_eal/linuxapp/igb_uio Building modules, stage 2. MODPOST 1 modules == Build lib/librte_eal/linuxapp/eal CC eal.o In file included from /home/lwrt/build/dpdk/lib/librte_eal/linuxapp/eal/eal.c:55:0: /home/lwrt/build/dpdk/lib/librte_eal/linuxapp/eal/eal.c: In function ?eal_parse_base_virtaddr?: /home/lwrt/build/dpdk/i686-default-linuxapp-gcc/include/rte_common.h:133:22: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] RTE_PTR_ALIGN_FLOOR((typeof(ptr))RTE_PTR_ADD(ptr, (align) - 1), align) ^ /home/lwrt/build/dpdk/i686-default-linuxapp-gcc/include/rte_common.h:115:10: note: in definition of macro ?RTE_PTR_ALIGN_FLOOR? (typeof(ptr))rte_align_floor_int((uintptr_t)ptr, align) ^ /home/lwrt/build/dpdk/lib/librte_eal/linuxapp/eal/eal.c:566:9: note: in expansion of macro ?RTE_PTR_ALIGN_CEIL? addr = RTE_PTR_ALIGN_CEIL(addr, RTE_PGSIZE_2M); ^ /home/lwrt/build/dpdk/i686-default-linuxapp-gcc/include/rte_common.h:133:22: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] RTE_PTR_ALIGN_FLOOR((typeof(ptr))RTE_PTR_ADD(ptr, (align) - 1), align) ^ /home/lwrt/build/dpdk/i686-default-linuxapp-gcc/include/rte_common.h:115:46: note: in definition of macro ?RTE_PTR_ALIGN_FLOOR? (typeof(ptr))rte_align_floor_int((uintptr_t)ptr, align) ^ /home/lwrt/build/dpdk/lib/librte_eal/linuxapp/eal/eal.c:566:9: note: in expansion of macro ?RTE_PTR_ALIGN_CEIL? addr = RTE_PTR_ALIGN_CEIL(addr, RTE_PGSIZE_2M); ^ cc1: all warnings being treated as errors make[6]: *** [eal.o] Error 1 make[5]: *** [eal] Error 2 make[4]: *** [linuxapp] Error 2 make[3]: *** [librte_eal] Error 2 make[2]: *** [lib] Error 2 make[1]: *** [all] Error 2 make: *** [all] Error 2 Is this a known problem? Best regards Mats Liljegren
[dpdk-dev] How to debug packet sends to virtual functions
>>> It's the latter, i.e. two primary processes running DPDK on the same VM but >>> different VF devices. The VF devices belongs to the same physical devices >>> though. >> >> I'll try that as well. > > If you need more help with how we set things up, I'll be happy to help you. I found the bug, it was an illegal arp entry that I had created causing the problem. So this case has been solved. I re-checked this ifconfig issue, and now I can get it to work without ifconfig in the guest. Not sure why it didn't work without it previously, but apparently, there is something else I did that made things work. Sorry about the fuzz, but thanks for your effort! Regards Mats Liljegren
[dpdk-dev] How to debug packet sends to virtual functions
Hi Anatoly, >> The guest starts with loading the igbvf kernel driver, uses ifconfig >> up, >> then loads and binds to igb_uio. After that, DPDK works. If I skip any step >> here it doesn't work. Skipping "ifconfig" step resulted in packets being >> received but I couldn't send them, they just got queued up but was never >> sent. >> > > Hm, never seen this before. Apologies if you have already mentioned this, but > what NIC are you using? I'll see if I can replicate your issues. Not sure if I've mentioned it, but it is a quad I350. Port 0 and 3 on host only, and 1 and 2 are being used by DPDK in a guest. The challenge was to make a ping go from port 0 to 3 and back, via a somewhat modified DPDK l2fwd example. Port 1 and 2 had two virtual functions each, and each such pair was given to an instance of this l2fwd example. >> It's the latter, i.e. two primary processes running DPDK on the same VM but >> different VF devices. The VF devices belongs to the same physical devices >> though. > > I'll try that as well. If you need more help with how we set things up, I'll be happy to help you. Regards Mats Liljegren
[dpdk-dev] How to debug packet sends to virtual functions
>> I finally got things working. I apparently missed an "ifconfig up" in >> the >> guest, before starting dpdk. I'm still confused why this would be needed. Is >> dpdk unable to do a full initialization of the virtual function from the >> guest? >> > You wouldn't be able to call ifconfig on your VF device if you have bound > your guest VF device to igb_uio driver. Have you bound your VF device to > igb_uio? Or you have enabled automatic port unbinding (which is disabled by > default in recent releases)? The guest starts with loading the igbvf kernel driver, uses ifconfig up, then loads and binds to igb_uio. After that, DPDK works. If I skip any step here it doesn't work. Skipping "ifconfig" step resulted in packets being received but I couldn't send them, they just got queued up but was never sent. >> While I got one instance of DPDK running, I got a problem when starting two >> instances of DPDK running against different virtual functions. These virtual >> functions stems from the same physical interfaces. >> >> Starting them one or the other works fine. I have to adapt my static arp >> entries since they have different MAC addresses, but this is only expected. >> When starting them both however, I receive no packets. If one is running >> and currently processing packets, it will stop doing so the instance the >> second >> instance starts. >> > Are you referring to two different VM's each having a separate VF device, or > are you trying to run two primary processes on one VM with different VF > devices? It's the latter, i.e. two primary processes running DPDK on the same VM but different VF devices. The VF devices belongs to the same physical devices though. Regards Mats Liljegren
[dpdk-dev] How to debug packet sends to virtual functions
On Tue, Feb 4, 2014 at 2:40 PM, Burakov, Anatoly wrote: > Hi Mats > > Technically, you can use igb_uio on the host as well (DPDK PMD supports > creating virtual devices since at least release 1.5.0), it's just that you'll > have to set everything up yourself inside your host DPDK application (you > can't use "ip net" to set up VF devices if you're using DPDK drivers). > Unfortunately, I'm not familiar enough with that part of the code to comment > on what exactly you should do to make it work with igb_uio, but I can > certainly ask around if you want. > > I also noticed that you are running KVM with virsh. We always run our VM's by > passing QEMU command line directly, without using virsh, so unfortunately I > cannot be of much help here as I'm not familiar with virsh. Your best bet > would be to get whatever parameters virsh passes to QEMU and modify them to > suit your needs and according to DPDK documentation. Hi Anatoly, I finally got things working. I apparently missed an "ifconfig up" in the guest, before starting dpdk. I'm still confused why this would be needed. Is dpdk unable to do a full initialization of the virtual function from the guest? I'm also curious whether this is how it is intended to work, or am I experiencing a strange type of bug? If not, is it documented that ifconfig is needed in the guest? I thought I've read all documentation thoroughly, but sometimes it is easy to miss the obvious anyway... While I got one instance of DPDK running, I got a problem when starting two instances of DPDK running against different virtual functions. These virtual functions stems from the same physical interfaces. Starting them one or the other works fine. I have to adapt my static arp entries since they have different MAC addresses, but this is only expected. When starting them both however, I receive no packets. If one is running and currently processing packets, it will stop doing so the instance the second instance starts. Anyone knowing what could cause such behavior? Regards Mats Liljegren
[dpdk-dev] How to debug packet sends to virtual functions
Hi Anatoly, Just realized that the instructions gave two alternatives for the host: DPDK igb_uio or ixgbe. It was the former, DPDK igb_uio that I was referring to as being the non-working alternative. Loading ixgbe enables me to set the mac addresses. This is also what I tried previously, when I both got ixgbevf working in the guest as well as receive to work in DPDK in the guest. The problem is making transmit work in DPDK in the guest. Regards Mats On Tue, Feb 4, 2014 at 12:52 PM, Burakov, Anatoly wrote: >> -Original Message- >> From: Mats Liljegren [mailto:liljegren.mats2 at gmail.com] >> Sent: Tuesday, February 04, 2014 11:48 AM >> To: Burakov, Anatoly >> Cc: dev at dpdk.org >> Subject: Re: [dpdk-dev] How to debug packet sends to virtual functions >> >> Hi Anatoly, >> >> Thanks for the clarification. Then I can't use those instructions, which is >> what I >> was starting to suspect. I have to find another way of getting DPDK running >> in >> the guest then. Using ixgbevf in the guest works fine, though. >> >> Regards >> Mats >> > > Hi Mats > > Let me clarify - do you run DPDK on both host and guest? E.g. you load > igb_uio on the host as well? If not, I don't see why you can't use those > instructions - you're creating VF devices on the host anyway, and that's > where you should set their MAC addresses. > > Best regards, > Anatoly Burakov > DPDK SW Engineer > > -- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 Business address: Dromore House, East Park, > Shannon, Co. Clare > >
[dpdk-dev] How to debug packet sends to virtual functions
Hi Anatoly, Thanks for the clarification. Then I can't use those instructions, which is what I was starting to suspect. I have to find another way of getting DPDK running in the guest then. Using ixgbevf in the guest works fine, though. Regards Mats On Tue, Feb 4, 2014 at 12:21 PM, Burakov, Anatoly wrote: >> -Original Message- >> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Mats Liljegren >> Sent: Tuesday, February 04, 2014 10:45 AM >> To: jigsaw >> Cc: dev at dpdk.org >> Subject: Re: [dpdk-dev] How to debug packet sends to virtual functions >> >> Hi Qinglai, >> >> Thanks for the response! >> >> My previous attempt was with ixgbe loaded in the host. I also needed to load >> ixgbevf, but this seems to be because of a short-coming in libvirt. Maybe >> loading ixgbevf and then unbind it when running the guest is what causes my >> problems. I could receive packets with this setup, but not transmit. I used >> extra debug to syslog, and it showed that packets was placed in the transmit >> queue, but these packets was never sent. >> >> I'll see if I can get this working without loading ixgbevf in the host. >> >> What instructions did you follow to get this working? Did you do any >> deviation from the instructions? >> >> Regards >> Mats >> >> On Tue, Feb 4, 2014 at 11:26 AM, jigsaw wrote: >> > Hi Mats, >> > >> > I've tried vf with 82599EB and it works fine. As long as the VF is >> > visible in guest, DPDK's VF driver should work just as ixgbevf, which >> > shares more or less the same code. >> > >> > I don't understand why you would expect DPDK at guest to work as VF, >> > while the host has no ixgbe loaded. >> > To make further debug I'd suggest compile ixgbe driver with your own >> > syslogs. At least you will be able to see the signalling between vf >> > and ixgbe drivers. >> > >> > -Qinglai >> > >> > >> > On Tue, Feb 4, 2014 at 12:08 PM, Mats Liljegren >> > wrote: >> >> This is my fourth mail in my desperate attempt to get DPDK running in >> >> KVM and no comments so far, not even any "it works for me". Am I the >> >> only one crazy enough to believe that this can be done? >> >> >> >> Anyway, out of desperation I tried to get it running without having >> >> ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed >> >> the instructions in the Programmer's Guide, chapter "Setting Up a KVM >> >> Virtual Machine Monitor", using the PMD version of the instructions. >> >> >> >> I get as far as being able to see my four virtual functions in the >> >> guest using "lspci". But starting the DPDK application gives me the >> >> following error: >> >> >> >> PMD:The MAC address is not valid. >> >> The most likely cause of this error is that the VM host >> >> has not assigned a valid MAC address to this VF device. >> >> Please consult the DPDK Release Notes (FAQ section) for >> >> a possible solution to this problem. >> >> >> >> This may be true, but without any kernel modules loaded, how am I >> >> supposed to change any MAC addresses? Can this be done from within >> >> DPDK? >> >> >> >> As a side-note, I did try to load ixgbevf in the guest, but it >> >> produced no interfaces. There was no error messages in the syslog >> >> though. >> >> >> >> Is it possible to get X540 working in a guest or should I switch hardware? >> >> >> >> Since the instructions assumes I know the command line to KVM to >> >> start my guest (which I do not), I cannot followed them precisely. I >> >> use virsh and XML file, and maybe I've misunderstood how to translate >> >> the pci-assign parameter to XML code. I currently use a >> >> entry, but I've also tried . Neither has >> >> been working for me so far, though the version got me as >> >> far as being able to receive packets at least, but not transmitting. >> >> >> >> Regards >> >> Mats >> >> >> >> >> >> On Mon, Feb 3, 2014 at 12:13 PM, Mats Liljegren >> >> wrote: >> >>> Never mind, I was hit by the infamous MAC spoofing... I got it >> >>> working on both the host and the guest using ixgbevf driver, so >> >>> apparently the c
[dpdk-dev] How to debug packet sends to virtual functions
Hi Qinglai, How did you configure the PCI passthrough in KVM? Did you use command line parameter as described in the instructions, or do you use virsh XML files, or maybe using virt-manager? The steps that works best (i.e. can receive but not transmit) are: 1) sudo modprobe ixgbe max_vfs=2 2) sudo ip link set em1 vf 0 mac 3) sudo ifconfig em1 up 4) sudo modprobe pci-stub 5) echo "8086 1515" | sudo tee /sys/bus/pci/drivers/pci-stub/new_id 6) echo ":02:10.0" | sudo tee /sys/bus/pci/devices/\:02\:10.0/driver/unbind 7) echo ":02:10.0" | sudo tee /sys/bus/pci/drivers/pci-stub/bind 8) virsh create vm vm.xml The guest then does: 5) sudo modprobe uio 6) sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio :02:10.0 7) sudo $RTE_SDK/tools/pci_unbind.py -b igb_uio 8) sudo The virsh xml file has the following entry for the virtual function: I actually have two virtual functions and two physical functions, but I just repeat above for each. Regards Mats On Tue, Feb 4, 2014 at 12:14 PM, jigsaw wrote: > Hi Mats, > > I didn't have any deviation. What I did is just loading ixgbe (with > extra params for vf, as you mentioned in your first email), > and DPDK is up and running in guest. Of course I also followed section > 8.10 of the DPDK release notes. > > I can switch between DPDK and ixgebvf in guest at runtime and > everything works fine. > > Sorry I can't help to debug coz I have only 82599EB at hand. > > -Qinglai > > On Tue, Feb 4, 2014 at 12:45 PM, Mats Liljegren > wrote: >> Hi Qinglai, >> >> Thanks for the response! >> >> My previous attempt was with ixgbe loaded in the host. I also needed >> to load ixgbevf, but this seems to be because of a short-coming in >> libvirt. Maybe loading ixgbevf and then unbind it when running the >> guest is what causes my problems. I could receive packets with this >> setup, but not transmit. I used extra debug to syslog, and it showed >> that packets was placed in the transmit queue, but these packets was >> never sent. >> >> I'll see if I can get this working without loading ixgbevf in the host. >> >> What instructions did you follow to get this working? Did you do any >> deviation from the instructions? >> >> Regards >> Mats >> >> On Tue, Feb 4, 2014 at 11:26 AM, jigsaw wrote: >>> Hi Mats, >>> >>> I've tried vf with 82599EB and it works fine. As long as the VF is >>> visible in guest, DPDK's VF driver should work just as ixgbevf, which >>> shares more or less the same code. >>> >>> I don't understand why you would expect DPDK at guest to work as VF, >>> while the host has no ixgbe loaded. >>> To make further debug I'd suggest compile ixgbe driver with your own >>> syslogs. At least you will be able to see the signalling between vf >>> and ixgbe drivers. >>> >>> -Qinglai >>> >>> >>> On Tue, Feb 4, 2014 at 12:08 PM, Mats Liljegren >>> wrote: >>>> This is my fourth mail in my desperate attempt to get DPDK running in >>>> KVM and no comments so far, not even any "it works for me". Am I the >>>> only one crazy enough to believe that this can be done? >>>> >>>> Anyway, out of desperation I tried to get it running without having >>>> ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed >>>> the instructions in the Programmer's Guide, chapter "Setting Up a KVM >>>> Virtual Machine Monitor", using the PMD version of the instructions. >>>> >>>> I get as far as being able to see my four virtual functions in the >>>> guest using "lspci". But starting the DPDK application gives me the >>>> following error: >>>> >>>> PMD:The MAC address is not valid. >>>> The most likely cause of this error is that the VM host >>>> has not assigned a valid MAC address to this VF device. >>>> Please consult the DPDK Release Notes (FAQ section) for >>>> a possible solution to this problem. >>>> >>>> This may be true, but without any kernel modules loaded, how am I >>>> supposed to change any MAC addresses? Can this be done from within >>>> DPDK? >>>> >>>> As a side-note, I did try to load ixgbevf in the guest, but it >>>> produced no interfaces. There was no error messages in the syslog >>>> though. >>>> >>>> Is it possible to get X540 working in a guest or should I switch hardware?
[dpdk-dev] How to debug packet sends to virtual functions
Hi Qinglai, Thanks for the response! My previous attempt was with ixgbe loaded in the host. I also needed to load ixgbevf, but this seems to be because of a short-coming in libvirt. Maybe loading ixgbevf and then unbind it when running the guest is what causes my problems. I could receive packets with this setup, but not transmit. I used extra debug to syslog, and it showed that packets was placed in the transmit queue, but these packets was never sent. I'll see if I can get this working without loading ixgbevf in the host. What instructions did you follow to get this working? Did you do any deviation from the instructions? Regards Mats On Tue, Feb 4, 2014 at 11:26 AM, jigsaw wrote: > Hi Mats, > > I've tried vf with 82599EB and it works fine. As long as the VF is > visible in guest, DPDK's VF driver should work just as ixgbevf, which > shares more or less the same code. > > I don't understand why you would expect DPDK at guest to work as VF, > while the host has no ixgbe loaded. > To make further debug I'd suggest compile ixgbe driver with your own > syslogs. At least you will be able to see the signalling between vf > and ixgbe drivers. > > -Qinglai > > > On Tue, Feb 4, 2014 at 12:08 PM, Mats Liljegren > wrote: >> This is my fourth mail in my desperate attempt to get DPDK running in >> KVM and no comments so far, not even any "it works for me". Am I the >> only one crazy enough to believe that this can be done? >> >> Anyway, out of desperation I tried to get it running without having >> ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed >> the instructions in the Programmer's Guide, chapter "Setting Up a KVM >> Virtual Machine Monitor", using the PMD version of the instructions. >> >> I get as far as being able to see my four virtual functions in the >> guest using "lspci". But starting the DPDK application gives me the >> following error: >> >> PMD:The MAC address is not valid. >> The most likely cause of this error is that the VM host >> has not assigned a valid MAC address to this VF device. >> Please consult the DPDK Release Notes (FAQ section) for >> a possible solution to this problem. >> >> This may be true, but without any kernel modules loaded, how am I >> supposed to change any MAC addresses? Can this be done from within >> DPDK? >> >> As a side-note, I did try to load ixgbevf in the guest, but it >> produced no interfaces. There was no error messages in the syslog >> though. >> >> Is it possible to get X540 working in a guest or should I switch hardware? >> >> Since the instructions assumes I know the command line to KVM to start >> my guest (which I do not), I cannot followed them precisely. I use >> virsh and XML file, and maybe I've misunderstood how to translate the >> pci-assign parameter to XML code. I currently use a entry, >> but I've also tried . Neither has been >> working for me so far, though the version got me as far as >> being able to receive packets at least, but not transmitting. >> >> Regards >> Mats >> >> >> On Mon, Feb 3, 2014 at 12:13 PM, Mats Liljegren >> wrote: >>> Never mind, I was hit by the infamous MAC spoofing... I got it working >>> on both the host and the guest using ixgbevf driver, so apparently the >>> cables are correctly attached. >>> >>> Using DPDK is still no-go. It can receive packets, but when sending >>> the packets the function returns success, but the driver reports >>> nothing (i.e. no errors, no sent packets, no nothing, except for >>> received packets of course). >>> >>> What could cause this behavior? >>> >>> Regards >>> Mats >>> >>> On Fri, Jan 31, 2014 at 7:30 PM, Mats Liljegren >>> wrote: >>>> I have a follow-up on this: >>>> >>>> ixgbe version 3.13.10-k >>>> ixgbevf version 2.7.12-k >>>> >>>> (These are what was provided by Ubuntu 13.10) >>>> >>>> I tried the following sequence on the host, before starting the guest: >>>> 1) sudo rmmod ixgbe >>>> 2) sudo modprobe ixgbe max_vfs=2 >>>> 3) sudo ifconfig em1 up # This is the physical function >>>> 4) sudo ifconfig em1_0 192.168.2.2 # This is the virtual function >>>> 5) ping 192.168.2.1 >>>> >>>> I can see that the ping request reaches its target, and a reply is >>>> sent back. But this reply is not received by the ping shell command. >>>> >>>>
[dpdk-dev] How to debug packet sends to virtual functions
This is my fourth mail in my desperate attempt to get DPDK running in KVM and no comments so far, not even any "it works for me". Am I the only one crazy enough to believe that this can be done? Anyway, out of desperation I tried to get it running without having ixgbe or ixgbevf kernel modules loaded in host nor guest. I followed the instructions in the Programmer's Guide, chapter "Setting Up a KVM Virtual Machine Monitor", using the PMD version of the instructions. I get as far as being able to see my four virtual functions in the guest using "lspci". But starting the DPDK application gives me the following error: PMD:The MAC address is not valid. The most likely cause of this error is that the VM host has not assigned a valid MAC address to this VF device. Please consult the DPDK Release Notes (FAQ section) for a possible solution to this problem. This may be true, but without any kernel modules loaded, how am I supposed to change any MAC addresses? Can this be done from within DPDK? As a side-note, I did try to load ixgbevf in the guest, but it produced no interfaces. There was no error messages in the syslog though. Is it possible to get X540 working in a guest or should I switch hardware? Since the instructions assumes I know the command line to KVM to start my guest (which I do not), I cannot followed them precisely. I use virsh and XML file, and maybe I've misunderstood how to translate the pci-assign parameter to XML code. I currently use a entry, but I've also tried . Neither has been working for me so far, though the version got me as far as being able to receive packets at least, but not transmitting. Regards Mats On Mon, Feb 3, 2014 at 12:13 PM, Mats Liljegren wrote: > Never mind, I was hit by the infamous MAC spoofing... I got it working > on both the host and the guest using ixgbevf driver, so apparently the > cables are correctly attached. > > Using DPDK is still no-go. It can receive packets, but when sending > the packets the function returns success, but the driver reports > nothing (i.e. no errors, no sent packets, no nothing, except for > received packets of course). > > What could cause this behavior? > > Regards > Mats > > On Fri, Jan 31, 2014 at 7:30 PM, Mats Liljegren > wrote: >> I have a follow-up on this: >> >> ixgbe version 3.13.10-k >> ixgbevf version 2.7.12-k >> >> (These are what was provided by Ubuntu 13.10) >> >> I tried the following sequence on the host, before starting the guest: >> 1) sudo rmmod ixgbe >> 2) sudo modprobe ixgbe max_vfs=2 >> 3) sudo ifconfig em1 up # This is the physical function >> 4) sudo ifconfig em1_0 192.168.2.2 # This is the virtual function >> 5) ping 192.168.2.1 >> >> I can see that the ping request reaches its target, and a reply is >> sent back. But this reply is not received by the ping shell command. >> >> Why? >> >> Regards, >> Mats >> >> On Wed, Jan 29, 2014 at 6:56 PM, Mats Liljegren >> wrote: >>> I'm trying to get a modified version of the l2fwd example running, and >>> have problems with packets being silently thrown away. I can receive >>> packets, and my printf's indicates that the packets are being sent to >>> the correct port, using correct MAC address as source address. And >>> still, the packets are lost. >>> >>> Since the port is a virtual function, it seems like I cannot use >>> tcpdump on it to see the network traffic. There is nothing coming out >>> of the cable (activity light not flashing, the receiving end running >>> tcpdump does not see any traffic). >>> >>> I'm using two X540 with two virtual functions each. The application >>> runs in a KVM/Qemu environmen. >>> >>> Any suggestions how to debug this? >>> >>> Regards, >>> Mats
[dpdk-dev] How to debug packet sends to virtual functions
Never mind, I was hit by the infamous MAC spoofing... I got it working on both the host and the guest using ixgbevf driver, so apparently the cables are correctly attached. Using DPDK is still no-go. It can receive packets, but when sending the packets the function returns success, but the driver reports nothing (i.e. no errors, no sent packets, no nothing, except for received packets of course). What could cause this behavior? Regards Mats On Fri, Jan 31, 2014 at 7:30 PM, Mats Liljegren wrote: > I have a follow-up on this: > > ixgbe version 3.13.10-k > ixgbevf version 2.7.12-k > > (These are what was provided by Ubuntu 13.10) > > I tried the following sequence on the host, before starting the guest: > 1) sudo rmmod ixgbe > 2) sudo modprobe ixgbe max_vfs=2 > 3) sudo ifconfig em1 up # This is the physical function > 4) sudo ifconfig em1_0 192.168.2.2 # This is the virtual function > 5) ping 192.168.2.1 > > I can see that the ping request reaches its target, and a reply is > sent back. But this reply is not received by the ping shell command. > > Why? > > Regards, > Mats > > On Wed, Jan 29, 2014 at 6:56 PM, Mats Liljegren > wrote: >> I'm trying to get a modified version of the l2fwd example running, and >> have problems with packets being silently thrown away. I can receive >> packets, and my printf's indicates that the packets are being sent to >> the correct port, using correct MAC address as source address. And >> still, the packets are lost. >> >> Since the port is a virtual function, it seems like I cannot use >> tcpdump on it to see the network traffic. There is nothing coming out >> of the cable (activity light not flashing, the receiving end running >> tcpdump does not see any traffic). >> >> I'm using two X540 with two virtual functions each. The application >> runs in a KVM/Qemu environmen. >> >> Any suggestions how to debug this? >> >> Regards, >> Mats
[dpdk-dev] How to debug packet sends to virtual functions
I have a follow-up on this: ixgbe version 3.13.10-k ixgbevf version 2.7.12-k (These are what was provided by Ubuntu 13.10) I tried the following sequence on the host, before starting the guest: 1) sudo rmmod ixgbe 2) sudo modprobe ixgbe max_vfs=2 3) sudo ifconfig em1 up # This is the physical function 4) sudo ifconfig em1_0 192.168.2.2 # This is the virtual function 5) ping 192.168.2.1 I can see that the ping request reaches its target, and a reply is sent back. But this reply is not received by the ping shell command. Why? Regards, Mats On Wed, Jan 29, 2014 at 6:56 PM, Mats Liljegren wrote: > I'm trying to get a modified version of the l2fwd example running, and > have problems with packets being silently thrown away. I can receive > packets, and my printf's indicates that the packets are being sent to > the correct port, using correct MAC address as source address. And > still, the packets are lost. > > Since the port is a virtual function, it seems like I cannot use > tcpdump on it to see the network traffic. There is nothing coming out > of the cable (activity light not flashing, the receiving end running > tcpdump does not see any traffic). > > I'm using two X540 with two virtual functions each. The application > runs in a KVM/Qemu environmen. > > Any suggestions how to debug this? > > Regards, > Mats
[dpdk-dev] How to debug packet sends to virtual functions
I'm trying to get a modified version of the l2fwd example running, and have problems with packets being silently thrown away. I can receive packets, and my printf's indicates that the packets are being sent to the correct port, using correct MAC address as source address. And still, the packets are lost. Since the port is a virtual function, it seems like I cannot use tcpdump on it to see the network traffic. There is nothing coming out of the cable (activity light not flashing, the receiving end running tcpdump does not see any traffic). I'm using two X540 with two virtual functions each. The application runs in a KVM/Qemu environmen. Any suggestions how to debug this? Regards, Mats
[dpdk-dev] [PATCH v3 2/2] pcap: Fill in if_index field for rte_eth_dev_info_get()
I can make new patches with the requested new titles. I found a compiler warning caused by my code that I can remove at the same time... struct args_dict is found in rte_eth_pcap_arg_parser.h and is a dictionary for command line parameters. /Mats On Fri, Jan 17, 2014 at 6:48 PM, Thomas Monjalon wrote: > what about this title? > "pcap: save if_index of the bound device" > >> From: Mats Liljegren >> >> Signed-off-by: Mats Liljegren > > Could you provide a little explanation, especially about struct args_dict ? > > The code seems OK. > Thanks > -- > Thomas
[dpdk-dev] No information about needed pcap version
You could let PCAP_SEND be always 1 in rte_eth_pcap.h and then document that for libpcap < 1.0.0 you need to change PCAP_SEND to 0 to avoid link error for pcap_sendpacket(). Regards Mats On Thu, Jan 9, 2014 at 2:47 PM, David Marchand wrote: > Hello, > > Indeed, this check on a define is wrong. > > pcap.h does not seem to have information on library version (nor a #define > linked to pcap_sendpacket functionality). > > We need to think of a better check but, for now, you can try to #define > PCAP_SEND to 1 in lib/librte_pmd_pcap/rte_eth_pcap.h. > pcap_sendpacket() is undefined for really old versions of libpcap (I think < > 1.0.0). > I suppose we might need something like configure (trying to link against > pcap lib and see if pcap_sendpacket symbol is defined). > > Or we can revert 4fc6677d995bb46ddf155ee08a215f41e5ecbfe8 and declare > libpcap < X.X.X unsupported. > > > -- > David Marchand > > > On Thu, Jan 9, 2014 at 12:59 PM, Mats Liljegren > wrote: >> >> In file lib/librte_pmd_pcap/rte_eth_pcap.h there is a test to see >> whether function pcap_sendpacket is a macro or not. If not, send is >> not supported using pcap. >> >> My pcap library do have the function, but not defined as a macro. >> >> I'm using libpcap-dev version 1.4.0, but I couldn't find any >> information in the manuals what version I need. >> >> Which version do I need to make this work? >> >> Regards >> Mats > >
[dpdk-dev] No information about needed pcap version
In file lib/librte_pmd_pcap/rte_eth_pcap.h there is a test to see whether function pcap_sendpacket is a macro or not. If not, send is not supported using pcap. My pcap library do have the function, but not defined as a macro. I'm using libpcap-dev version 1.4.0, but I couldn't find any information in the manuals what version I need. Which version do I need to make this work? Regards Mats
[dpdk-dev] [PATCH 1/2] Introduce if_index field to struct rte_eth_dev_info
On Thu, Jan 9, 2014 at 7:30 AM, Stephen Hemminger
wrote:
> Technically in Linux ifindex is unsigned 32 bit value. And 0 is
> reserved as a marker.
> Therefore why not use that semantic.
>
> On Wed, Jan 8, 2014 at 1:46 AM, Mats Liljegren
> wrote:
>> This field is intended for pcap to describe the name of the interface
>> as known to Linux. It is an interface index, but can be translated into
>> an interface name using if_indextoname() function.
>>
>> When using pcap, interrupt affinity becomes important, and this field
>> gives the application a chance to ensure that interrupt affinity is set
>> to the lcore handling the device.
>>
>> Signed-off-by: Mats Liljegren
>> ---
>> lib/librte_ether/rte_ethdev.c | 1 +
>> lib/librte_ether/rte_ethdev.h | 1 +
>> 2 files changed, 2 insertions(+)
>>
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
>> index 859ec92..38c1ea1 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -1037,6 +1037,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct
>> rte_eth_dev_info *dev_info)
>> /* Default device offload capabilities to zero */
>> dev_info->rx_offload_capa = 0;
>> dev_info->tx_offload_capa = 0;
>> + dev_info->if_index = -1;
>> FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
>> (*dev->dev_ops->dev_infos_get)(dev, dev_info);
>> dev_info->pci_dev = dev->pci_dev;
>> diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
>> index 302d378..5b80e5d 100644
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -787,6 +787,7 @@ struct rte_eth_conf {
>> struct rte_eth_dev_info {
>> struct rte_pci_device *pci_dev; /**< Device PCI information. */
>> const char *driver_name; /**< Device Driver name. */
>> + int if_index; /**< Index to bounded host interface, or -1 if
>> none. Use if_indextoname() to translate into an interface name. */
>> uint32_t min_rx_bufsize; /**< Minimum size of RX buffer. */
>> uint32_t max_rx_pktlen; /**< Maximum configurable length of RX pkt.
>> */
>> uint16_t max_rx_queues; /**< Maximum number of RX queues. */
>> --
>> 1.8.3.2
You're right. I'm too used to int and -1...
I'll prepare new patches.
[dpdk-dev] [PATCH 2/2] pcap: Fill in if_index field for rte_eth_dev_info_get() calls
Signed-off-by: Mats Liljegren
---
lib/librte_pmd_pcap/rte_eth_pcap.c | 39 ++
lib/librte_pmd_pcap/rte_eth_pcap.h | 6 --
2 files changed, 35 insertions(+), 10 deletions(-)
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c
b/lib/librte_pmd_pcap/rte_eth_pcap.c
index 87d1306..ebd16f3 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -86,6 +86,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
+ int if_index;
+
struct pcap_rx_queue rx_queue[RTE_PMD_RING_MAX_RX_RINGS];
struct pcap_tx_queue tx_queue[RTE_PMD_RING_MAX_TX_RINGS];
};
@@ -300,6 +302,7 @@ eth_dev_info(struct rte_eth_dev *dev,
{
struct pmd_internals *internals = dev->data->dev_private;
dev_info->driver_name = drivername;
+ dev_info->if_index = internals->if_index;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t) -1;
dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
@@ -543,10 +546,19 @@ rte_pmd_init_internals(const unsigned nb_rx_queues,
const unsigned nb_tx_queues,
const unsigned numa_node,
struct pmd_internals **internals,
- struct rte_eth_dev **eth_dev)
+ struct rte_eth_dev **eth_dev,
+ struct args_dict *dict)
{
struct rte_eth_dev_data *data = NULL;
struct rte_pci_device *pci_dev = NULL;
+ unsigned k_idx;
+ struct key_value *pair;
+
+ for (k_idx = 0; k_idx < dict->index; k_idx++) {
+ pair = >pairs[k_idx];
+ if (strstr(pair->key, ETH_PCAP_IFACE_ARG) != NULL)
+ break;
+ }
RTE_LOG(INFO, PMD,
"Creating pcap-backed ethdev on numa socket
%u\n", numa_node);
@@ -583,6 +595,15 @@ rte_pmd_init_internals(const unsigned nb_rx_queues,
(*internals)->nb_rx_queues = nb_rx_queues;
(*internals)->nb_tx_queues = nb_tx_queues;
+ if (k_idx == dict->index)
+ (*internals)->if_index = -1;
+ else
+ (*internals)->if_index = if_nametoindex(pair->value);
+
+ /* if_nametoindex() uses 0 as error report value, translate to -1 */
+ if ((*internals)->if_index == 0)
+ (*internals)->if_index = -1;
+
pci_dev->numa_node = numa_node;
data->dev_private = *internals;
@@ -612,7 +633,8 @@ rte_eth_from_pcaps_n_dumpers(pcap_t * const rx_queues[],
const unsigned nb_rx_queues,
pcap_dumper_t * const tx_queues[],
const unsigned nb_tx_queues,
- const unsigned numa_node)
+ const unsigned numa_node,
+ struct args_dict *dict)
{
struct pmd_internals *internals = NULL;
struct rte_eth_dev *eth_dev = NULL;
@@ -625,7 +647,7 @@ rte_eth_from_pcaps_n_dumpers(pcap_t * const rx_queues[],
return -1;
if (rte_pmd_init_internals(nb_rx_queues, nb_tx_queues, numa_node,
- , _dev) < 0)
+ , _dev, dict) < 0)
return -1;
for (i = 0; i < nb_rx_queues; i++) {
@@ -646,7 +668,8 @@ rte_eth_from_pcaps(pcap_t * const rx_queues[],
const unsigned nb_rx_queues,
pcap_t * const tx_queues[],
const unsigned nb_tx_queues,
- const unsigned numa_node)
+ const unsigned numa_node,
+ struct args_dict *dict)
{
struct pmd_internals *internals = NULL;
struct rte_eth_dev *eth_dev = NULL;
@@ -659,7 +682,7 @@ rte_eth_from_pcaps(pcap_t * const rx_queues[],
return -1;
if (rte_pmd_init_internals(nb_rx_queues, nb_tx_queues, numa_node,
- , _dev) < 0)
+ , _dev, dict) < 0)
return -1;
for (i = 0; i < nb_rx_queues; i++) {
@@ -707,7 +730,7 @@ rte_pmd_pcap_init(const char *name, const char *params)
if (ret < 0)
return -1;
- return rte_eth_from_pcaps(pcaps.pcaps, 1, pcaps.pcaps,
1, numa_node);
+ return rte_eth_from_pcaps(pcaps.pcaps, 1, pcaps.pcaps,
1, numa_node, );
}
/*
@@ -748,10 +771,10 @@ rte_pmd_pcap_init(const char *name, const char *params)
if (using_dumpers)
return rte_eth_from_pcaps_n_dumpers(pcaps.pcaps,
pcaps.num_of_rx,
- dumpers.dumpers, dumpers.num_of_tx, numa_node);
+ dumpers.dumpers, dumpers.num_of_tx,
numa_node, );
return rte_eth_from_pcaps(pcaps.pcaps, pcaps.num_of_rx, dumpers.pcaps,
- dumpers.num_of_tx, numa_node);
+ dumpers.num_of_tx, numa_node, );
}
diff --git a/lib/librte_pmd_pcap/rte_eth
[dpdk-dev] [PATCH 1/2] Introduce if_index field to struct rte_eth_dev_info
This field is intended for pcap to describe the name of the interface
as known to Linux. It is an interface index, but can be translated into
an interface name using if_indextoname() function.
When using pcap, interrupt affinity becomes important, and this field
gives the application a chance to ensure that interrupt affinity is set
to the lcore handling the device.
Signed-off-by: Mats Liljegren
---
lib/librte_ether/rte_ethdev.c | 1 +
lib/librte_ether/rte_ethdev.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 859ec92..38c1ea1 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -1037,6 +1037,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct
rte_eth_dev_info *dev_info)
/* Default device offload capabilities to zero */
dev_info->rx_offload_capa = 0;
dev_info->tx_offload_capa = 0;
+ dev_info->if_index = -1;
FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
(*dev->dev_ops->dev_infos_get)(dev, dev_info);
dev_info->pci_dev = dev->pci_dev;
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 302d378..5b80e5d 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -787,6 +787,7 @@ struct rte_eth_conf {
struct rte_eth_dev_info {
struct rte_pci_device *pci_dev; /**< Device PCI information. */
const char *driver_name; /**< Device Driver name. */
+ int if_index; /**< Index to bounded host interface, or -1 if
none. Use if_indextoname() to translate into an interface name. */
uint32_t min_rx_bufsize; /**< Minimum size of RX buffer. */
uint32_t max_rx_pktlen; /**< Maximum configurable length of RX pkt. */
uint16_t max_rx_queues; /**< Maximum number of RX queues. */
--
1.8.3.2
[dpdk-dev] [PATCH] RFC: Introduce host_ifname field to struct rte_eth_dev_info
I played around with pcap to compare it against "pure" DPDK user-space
driver. I realized that interrupt affinity suddenly becomes important,
since it uses interrupts. So I needed a way to set it.
The only place where I know which core will handle which pcap
interface, is within my DPDK application. But DPDK did not save the
host interface name anywhere, so it was nowhere to be found.
I created a patch for this, found below. Is this something that could
be upstreamed?
Best regards
Mats Liljegren
-- >8 --
>From 1b126c7f74680afae8a6426a197d77eab0f2b75e Mon Sep 17 00:00:00 2001
From: Mats Liljegren <mats.liljeg...@enea.com>
Date: Wed, 4 Dec 2013 16:31:59 +0100
Subject: [PATCH] Introduce host_ifname field to struct rte_eth_dev_info
This field is intended for pcap to describe the name of the interface
as known to Linux, e.g. eth2. When using pcap interrupt affinity
becomes important, and this field gives the application a chance
to ensure that interrupt affinity is set to to the lcore handling the
device.
An example how to use it:
static void
ensure_irq_affinity(unsigned port)
{
char path[255];
struct dirent *dirent;
DIR *dir;
const unsigned lcore = rte_lcore_id();
struct rte_eth_dev_info dev_info;
rte_eth_dev_info_get(port, _info);
if (dev_info.host_ifname[0] == '\0')
/* No host interface name */
return;
snprintf(path, sizeof (path), "/sys/class/net/%s/device/msi_irqs",
dev_info.host_ifname);
dir = opendir(path);
if (dir == NULL)
/* Can't do anything for this interface */
return;
for (dirent = readdir(dir); dirent != NULL; dirent = readdir(dir)) {
FILE *file;
if ((dirent->d_name[0] < '0') || (dirent->d_name[0] > '9'))
continue;
snprintf(path, sizeof (path), "/proc/irq/%s/smp_affinity",
dirent->d_name);
file = fopen(path, "w");
if (file == NULL)
rte_exit(EXIT_FAILURE,
"%s: Could not open file for writing: %s\n",
path, strerror(errno));
fprintf(file, "%x", 1 << lcore);
TRY(fclose(file));
printf("%s: Affinity set to 0x%x\n", dev_info.host_ifname, 1 << lcore);
}
closedir(dir);
}
Signed-off-by: Mats Liljegren
---
lib/librte_ether/rte_ethdev.c | 1 +
lib/librte_ether/rte_ethdev.h | 1 +
lib/librte_pmd_pcap/rte_eth_pcap.c | 42 ++
lib/librte_pmd_pcap/rte_eth_pcap.h | 7 +--
4 files changed, 41 insertions(+), 10 deletions(-)
diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index 675f247..bf126cf 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -1037,6 +1037,7 @@ rte_eth_dev_info_get(uint8_t port_id, struct
rte_eth_dev_info *dev_info)
/* Default device offload capabilities to zero */
dev_info->rx_offload_capa = 0;
dev_info->tx_offload_capa = 0;
+dev_info->host_ifname = "";
FUNC_PTR_OR_RET(*dev->dev_ops->dev_infos_get);
(*dev->dev_ops->dev_infos_get)(dev, dev_info);
dev_info->pci_dev = dev->pci_dev;
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 41268e1..8542911 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -786,6 +786,7 @@ struct rte_eth_conf {
struct rte_eth_dev_info {
struct rte_pci_device *pci_dev; /**< Device PCI information. */
const char *driver_name; /**< Device Driver name. */
+const char *host_ifname; /**< Name of interface as known to
Linux, or empty string. */
uint32_t min_rx_bufsize; /**< Minimum size of RX buffer. */
uint32_t max_rx_pktlen; /**< Maximum configurable length of RX pkt. */
uint16_t max_rx_queues; /**< Maximum number of RX queues. */
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c
b/lib/librte_pmd_pcap/rte_eth_pcap.c
index 08a944c..30bdd0c 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -87,6 +87,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
+char *iface_name;
+
struct pcap_rx_queue rx_queue[RTE_PMD_RING_MAX_RX_RINGS];
struct pcap_tx_queue tx_queue[RTE_PMD_RING_MAX_TX_RINGS];
};
@@ -284,6 +286,7 @@ eth_dev_info(struct rte_eth_dev *dev,
{
struct pmd_internals *internals = dev->data->dev_private;
dev_info->driver_name = drivername;
+dev_info->host_ifname = internals->iface_name;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t) -1;
dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
@@ -527,10 +530,22 @@ rte_pmd_init_internals(const unsigned nb_rx_queues,
const unsigned nb_tx_queues,
const unsigned numa_node,
struct pmd_internals **internals,
-
[dpdk-dev] Question: Can't make pcap and refcnt to match
On Wed, Dec 4, 2013 at 11:44 AM, Richardson, Bruce wrote: > [BR] Hi. Just so you know, a fix for this will be present in the Intel DPDK > 1.5.2 patch release from Intel, which should be publically available very > shortly. This fix we are releasing I also previously posted as a patch on > this list: http://www.dpdk.org/ml/archives/dev/2013-November/000855.html Thanks! I'll try that patch!
[dpdk-dev] Question: Can't make pcap and refcnt to match
Hi Bruce,
I made a dead simple patch that seems to fix my problem. Could you
check and see whether I'm on the right track here?
The patch is as follows:
-- >8 --
>From 901083b82c0e07f2535ee13f90e1a68c0f96602a Mon Sep 17 00:00:00 2001
From: Mats Liljegren <mats.liljeg...@enea.com>
Date: Tue, 3 Dec 2013 17:56:01 +0100
Subject: [PATCH] Fixed bug with receive causing segmentation violation and
lost buffers
A static list of 64 mbufs was being reused. This caused two errors:
1) If more than 64 buffer were requested in a single burst,
only the last 64 buffers are returned, the others are lost.
2) Application will free the mbuf being returned, but the receive
function will reuse the buffer anyway. If some other allocation
is done there is suddenly multiple writers for same mbuf.
The fix consists of replacing the reused buffers with an allocation
for each buffer being returned.
Signed-off-by: Mats Liljegren
---
lib/librte_pmd_pcap/rte_eth_pcap.c | 16 +++-
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/lib/librte_pmd_pcap/rte_eth_pcap.c
b/lib/librte_pmd_pcap/rte_eth_pcap.c
index 19d19b3..08a944c 100644
--- a/lib/librte_pmd_pcap/rte_eth_pcap.c
+++ b/lib/librte_pmd_pcap/rte_eth_pcap.c
@@ -118,30 +118,28 @@ eth_pcap_rx(void *queue,
struct pcap_pkthdr header;
const u_char *packet;
struct rte_mbuf *mbuf;
- static struct rte_mbuf *mbufs[RTE_ETH_PCAP_MBUFS] = { 0 };
struct pcap_rx_queue *pcap_q = queue;
uint16_t num_rx = 0;
if (unlikely(pcap_q->pcap == NULL || nb_pkts == 0))
return 0;
- if(unlikely(!mbufs[0]))
- for (i = 0; i < RTE_ETH_PCAP_MBUFS; i++)
- mbufs[i] = rte_pktmbuf_alloc(pcap_q->mb_pool);
-
/* Reads the given number of packets from the pcap file one by one
* and copies the packet data into a newly allocated mbuf to return.
*/
for (i = 0; i < nb_pkts; i++) {
- mbuf = mbufs[i % RTE_ETH_PCAP_MBUFS];
+ char * msg;
packet = pcap_next(pcap_q->pcap, );
if (unlikely(packet == NULL))
break;
+ mbuf = rte_pktmbuf_alloc(pcap_q->mb_pool);
if (unlikely(mbuf == NULL))
break;
- rte_memcpy(mbuf->pkt.data, packet, header.len);
- mbuf->pkt.data_len = (uint16_t)header.len;
- mbuf->pkt.pkt_len = mbuf->pkt.data_len;
+ msg = rte_pktmbuf_append(mbuf, header.len);
+ if (unlikely(msg == NULL)) {
+ rte_panic("MBuf too small, needed %u bytes\n",
header.len);
+ }
+ rte_memcpy(msg, packet, header.len);
bufs[i] = mbuf;
num_rx++;
}
--
1.8.3.2
On Tue, Nov 26, 2013 at 2:46 PM, Richardson, Bruce
wrote:
> Hi Mats,
>
> yes, you are right, there is an issue in the pcap driver that it is not
> allocating mbufs correctly. We are working on a fix.
>
> Regards,
> /Bruce
>
>> -Original Message-
>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Mats Liljegren
>> Sent: Tuesday, November 26, 2013 1:07 PM
>> To: dev at dpdk.org
>> Subject: [dpdk-dev] Question: Can't make pcap and refcnt to match
>>
>> I have had stability problems when using pcap in my little application. My
>> application is a simple benchmark applications that is trying to see how
>> much data I can send and receive.
>>
>> It has one lcore per NIC, where each lcore handles transmit and receive. On
>> the hardware, I make a loopback between two NICs, so the NICs are in
>> practice paired. I currently use 4 NICs and therefore 4 lcores. Port 0 sends
>> to
>> port 1 and vice versa. Port 2 send to port 3 and vice versa. One pair is
>> using
>> DPDK hardware driver against a dual
>> i350 NIC. The other pair is using pcap against two of the four on-board NICs.
>>
>> When enabling everything saying "DEBUG" in its name in the .config file, I
>> get the following error:
>>
>> PMD: rte_eth_dev_config_restore: port 1: MAC address array not
>> supported
>> PMD: rte_eth_promiscuous_disable: Function not supported
>> PMD: rte_eth_allmulticast_disable: Function not supported
>> Speed: 1 Mbps, full duplex
>> Port 1 up and running.
>> PMD: e1000_put_hw_semaphore_generic():
>> e1000_put_hw_semaphore_generic PANIC in rte_mbuf_sanity_check():
>> bad ref cnt
>> PANIC in rte_mbuf_sanity_check():
>> bad ref cnt
>> PMD: e1000_release_phy_82575(): e1000_release_phy_82575
>> PMD: e1000_release_swfw_sync_82575():
>> e1000_release_swfw_sync_82575
>> PMD: e1000_get_hw_semaphore_generic():
>> e1000
[dpdk-dev] Question: Can't make pcap and refcnt to match
I have had stability problems when using pcap in my little application. My application is a simple benchmark applications that is trying to see how much data I can send and receive. It has one lcore per NIC, where each lcore handles transmit and receive. On the hardware, I make a loopback between two NICs, so the NICs are in practice paired. I currently use 4 NICs and therefore 4 lcores. Port 0 sends to port 1 and vice versa. Port 2 send to port 3 and vice versa. One pair is using DPDK hardware driver against a dual i350 NIC. The other pair is using pcap against two of the four on-board NICs. When enabling everything saying "DEBUG" in its name in the .config file, I get the following error: PMD: rte_eth_dev_config_restore: port 1: MAC address array not supported PMD: rte_eth_promiscuous_disable: Function not supported PMD: rte_eth_allmulticast_disable: Function not supported Speed: 1 Mbps, full duplex Port 1 up and running. PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic PANIC in rte_mbuf_sanity_check(): bad ref cnt PANIC in rte_mbuf_sanity_check(): bad ref cnt PMD: e1000_release_phy_82575(): e1000_release_phy_82575 PMD: e1000_release_swfw_sync_82575(): e1000_release_swfw_sync_82575 PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fff776eefc0 hw_ring=0x7fff76830480 dma_addr=0x464630480 PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic PMD: To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fff776ece40 hw_ring=0x7fff76840500 dma_addr=0x464640500 PMD: eth_igb_start(): >> PMD: e1000_read_phy_reg_82580(): e1000_read_phy_reg_82580 PMD: e1000_acquire_phy_82575(): e1000_acquire_phy_82575 PMD: e1000_acquire_swfw_sync_82575(): e1000_acquire_swfw_sync_82575 PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic PMD: e1000_get_cfg_done_82575(): e1000_get_cfg_done_82575 PMD: e1000_put_hw_semaphore_generic(): e1000_put_hw_semaphore_generic PMD: e1000_read_phy_reg_mdic(): e1000_read_phy_reg_mdic 9: [/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x772a89cd]] 8: [/lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e) [0x7757df6e]] 7: [/home/mlil/dpdk-demo/build/enea-demo(eal_thread_loop+0x1b9) [0x492669]] 6: [/home/mlil/dpdk-demo/build/enea-demo() [0x4150bc]] 5: [/home/mlil/dpdk-demo/build/enea-demo() [0x414d0b]] 4: [/home/mlil/dpdk-demo/build/enea-demo() [0x4116ef]] 3: [/home/mlil/dpdk-demo/build/enea-demo(rte_mbuf_sanity_check+0xa7) [0x484707]] 2: [/home/mlil/dpdk-demo/build/enea-demo(__rte_panic+0xc1) [0x40f788]] 1: [/home/mlil/dpdk-demo/build/enea-demo(rte_dump_stack+0x18) [0x493f68]] PMD: e1000_release_phy_82575(): e1000_release_phy_82575 PMD: e1000_release_swfw_sync_82575(): e1000_release_swfw_sync_82575 PMD: e1000_get_hw_semaphore_generic(): e1000_get_hw_semaphore_generic I checked the source code for pcap, and in the file rte_eth_pcap.c, function eth_pcap_rx(), I make the following observation: It pre-allocates a number of mbufs (64 to be exact). It then fills these mbufs with data and returns them. The pre-allocation seems to only be done once, and then they are re-used. This confuses me. How does this work when more than 64 packets are requested? I see no safety checks for this. Aren't application supposed to call rte_pktmbuf_free() on the returned mbufs? If so, the pre-allocated mbufs will have been free'd as far as I can see and can therefore not be re-used. What am I missing here? Regards Mats
