Hey Christian! I think I found the problem! I just filled a bug report:
-- Many PMD driver options are disabled, including "BNX2X"! https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1559408 -- Let me know about what do you think! Cheers! Thiago On 19 March 2016 at 02:49, Martinx - ジェームズ <thiagocmarti...@gmail.com> wrote: > Oh, I forgot to mention that my /etc/dpdk/dpdk.conf have: > > --- > NR_1G_PAGES=2 > --- > > From what I'm seeing, my OpenvSwitch-DPDK is not using / seeing PMD. > > So, I'm researching about this but, I'll talk with you on Monday... > > Have a nice weekend Christian! > > Thanks, > Thiago > > On 18 March 2016 at 13:16, Martinx - ジェームズ <thiagocmarti...@gmail.com> > wrote: > >> Hello Christian! >> >> Lets make a checklist... >> >> >> 1- Packages "dpdk and openvswitch-switch-dpdk", installed; >> >> >> 2- /etc/default/grub have: >> >> GRUB_CMDLINE_LINUX_DEFAULT="quiet splash default_hugepagesz=1GB >> hugepagesz=1G hugepages=4" >> GRUB_CMDLINE_LINUX="console=ttyS0,115200n8 BOOTIF=01-b0-83-fe-d7-12-fc" >> >> Linux CMD Line: >> >> Command line: BOOT_IMAGE=/vmlinuz-4.4.0-13-generic >> root=/dev/mapper/xenial-1--vg-root ro console=ttyS0,115200n8 >> BOOTIF=01-b0-83-fe-d7-12-fc quiet splash default_hugepagesz=1GB >> hugepagesz=1G hugepages=4 vt.handoff=7 >> >> >> 3- /etc/dpdk/interfaces have: >> >> pci 0000:01:00.0 uio_pci_generic >> pci 0000:01:00.1 uio_pci_generic >> >> >> 4- /etc/default/openvswitch-switch have: >> >> DPDK_OPTS='--dpdk -c 0x1 -n 4 --socket-mem 1024,0' >> >> >> 5- /etc/alternatives/ovs-vswitchd: symbolic link to >> /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk >> >> >> 6- My CPU supports 1G Hugepages (it have the flag: pdpe1gb) >> >> >> 7- I am running most recent Xenial (4.4.0-13-generic), fully upgraded >> with "apt update ; apt full-upgrade ; reboot" >> >> >> 8- Huge pages on /proc/meminfo >> >> # grep Huge /proc/meminfo >> AnonHugePages: 26624 kB >> HugePages_Total: 2 >> HugePages_Free: 1 >> HugePages_Rsvd: 0 >> HugePages_Surp: 0 >> Hugepagesize: 1048576 kB >> >> >> >> 9- OVS process is running with DPDK options: >> >> --- >> ps xfa | grep dpdk >> 2634 ? S<s 0:00 ovs-vswit... --dpdk -c 0x1 -n 4 --socket-mem >> 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err >> -vfile:info --mlockall --no-chdir >> --log-file=/var/log/openvswitch/ovs-vswitchd.log >> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor >> 2635 ? S<Ll 0:00 \_ ovs-vswitchd --dpdk -c 0x1 -n 4 >> --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer >> -vsyslog:err -vfile:info --mlockall --no-chdir >> --log-file=/var/log/openvswitch/ovs-vswitchd.log >> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor >> --- >> >> >> 10- DPDK-Compatible Driver activated >> >> --- >> dpdk_nic_bind --status >> >> Network devices using DPDK-compatible driver >> ============================================ >> 0000:01:00.0 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' >> drv=uio_pci_generic unused=bnx2x >> 0000:01:00.1 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' >> drv=uio_pci_generic unused=bnx2x >> --- >> >> >> 11- OVS log have no "port 0" / "port 1" >> >> # service openvswitch-switch restart >> # journalctl | grep -i ovs: >> >> https://paste.ubuntu.com/15415651/ >> >> Also, I'm not seeing any "PMD" lines on my OVS logs! >> >> NOTE: after running: "service openvswitch-switch restart", it still have >> dpdk options (step 9 above still valid) >> >> >> 12- The, OVS commands also fails... >> >> # ovs-vsctl show >> b70c1e0a-20d7-4bdb-98db-467330b72d07 >> ovs_version: "2.5.0" >> >> # ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev >> # ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk >> ovs-vsctl: Error detected while setting up 'dpdk0'. See ovs-vswitchd log >> for details. >> >> --- >> >> I just don't know what I am missing, I already checked the configurations >> dozens of times... At least, I'm not alone seeing this problems, since >> there are more users on Intel's forum with same issue... >> >> Thank you for your help! I'm going to IRC now. >> >> Best, >> Thiago >> >> On 18 March 2016 at 10:22, Christian Ehrhardt < >> christian.ehrha...@canonical.com> wrote: >> >>> Hi Martin, >>> didn't find you on IRC today. >>> You might try to catch me later or on Monday if you like for faster >>> checks and comparison. >>> >>> I once more tried the ovs-dpdk setup. >>> While I'm blocked later on on other things I clearly can pass the stage >>> you are hanging on. >>> >>> I played a bit around and could cause the error you had by: >>> 1. not having any card on a dpdk compatible driver >>> 2. not enabling dpdk in ovs at all >>> >>> Could you please check your journalctl when starting openvswitch - it >>> should have a section like the following per card (ovs-)dpdk initializes: >>> >>> ovs-ctl[2155]: PMD: eth_ixgbe_dev_init(): MAC: 4, PHY: 3 >>> ovs-ctl[2155]: PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 >>> deviceID=0x1528 >>> horsea ovs-ctl[2155]: EAL: PCI device 0000:04:00.1 on NUMA socket 0 >>> horsea ovs-ctl[2155]: EAL: probe driver: 8086:1528 rte_ixgbe_pmd >>> horsea ovs-ctl[2155]: EAL: PCI memory mapped at 0x7f5d40204000 >>> horsea ovs-ctl[2155]: EAL: PCI memory mapped at 0x7f5d40404000 >>> horsea ovs-vswitchd[2189]: PMD: eth_ixgbe_dev_init(): MAC: 4, PHY: 3 >>> horsea ovs-vswitchd[2189]: PMD: eth_ixgbe_dev_init(): port 0 >>> vendorID=0x8086 deviceID=0x1528 >>> >>> You see "port 0" that will be what is "dpdk0" later on. >>> >>> When I e.g. have two devices in ugb_pci_generic and start OVS I get one >>> for port 0 and one for port 1. >>> Then I can later on use dpdk0 and dpdk1, but not e.g. dpdk2 >>> >>> >>> >>> Christian Ehrhardt >>> Software Engineer, Ubuntu Server >>> Canonical Ltd >>> >>> On Fri, Mar 11, 2016 at 10:17 PM, Martinx - ジェームズ < >>> thiagocmarti...@gmail.com> wrote: >>> >>>> Hey Christian, >>>> >>>> Nice job on fixing many DPDK package bugs! :-D >>>> >>>> Replying inline below... >>>> >>>> On 9 March 2016 at 04:17, Christian Ehrhardt < >>>> christian.ehrha...@canonical.com> wrote: >>>> >>>>> Thanks Jörg for making Martin and me aware. >>>>> It is kind of a close call, as much of that are new features, but you >>>>> are right - so I'll be moving this to the Ubuntu-sever mailing list. >>>>> >>>>> @Martin: >>>>> tl;dr - I suggest the following: >>>>> - you let me know if anything more comes back from the Intel forum post >>>>> >>>> >>>> Absolutely, I'll... >>>> >>>> >>>>> - short Term I'd ask you to reach out on IRC to me on #ubuntu-server - >>>>> my nick is "cpaelzer" (maybe we find something in an interactive session) >>>>> >>>> >>>> Sounds perfect! I'm there now, "tmartins" nick... >>>> >>>> >>>>> - somewhen in the next weeks I'll be back at openvswitch-dpdk. I'd >>>>> invite you to a IRC session or a hangout then (let me know what you would >>>>> prefer and I contact you once I'm ready) >>>>> >>>> >>>> That would be awesome! >>>> >>>> >>>>> >>>>> >>>>> Details: >>>>> The vfio-pci fix (and several others) are just waiting for the upload >>>>> to happen, if I don't run into any unexpected running blockers that should >>>>> be today. >>>>> >>>> >>>> Verified, it is working now. >>>> >>>> >>>>> Please let me know if you run into anything else with DPDK as I want >>>>> to clear as much as possible before release. >>>>> >>>> >>>> After latest release on Xenial, it looks very good! Both UIO and VFIO >>>> works without effort. >>>> >>>> I can easily switch between UIO and VFIO, that DPDK shows the NICs >>>> being used as "DPDK-Compatible drivers". >>>> >>>> Only OpenvSwitch+DPDK doesn't see any "dpdk0 | dpdk1" devices but, I >>>> know you guys are working into this... >>>> >>>> >>>>> Please feel free to just directly contact me about it - I'd suggest >>>>> IRC if that work for you. >>>>> We can move things to mailing lists whenever needed/applicable then. >>>>> >>>> >>>> Sounds great! >>>> >>>> >>>>> >>>>> Since last week I took a step back and work on more "low level" dpdk >>>>> testing and issues than the full setup with openvswitch-dpdk. >>>>> That helps the overall quality, but means my environment is currently >>>>> not useful to reproduce your case without too much reconfiguration. >>>>> >>>> >>>> That sounds like a good plan, I totally understand that... =) >>>> >>>> >>>>> >>>>> I highly appreciate to have someone in the Community working with me >>>>> on it. >>>>> >>>> >>>> You can count on me! I'm a big fan of Ubuntu... >>>> >>>> I met Mark in Vancouver on OpenStack summit, he is a great guy and it >>>> is easier now to see why Ubuntu is so cool! A lot of amazing people working >>>> here! ^_^ >>>> >>>> So I wanted to ask if you could be patient for a few days, once my Env >>>>> is working for openvswitch-dpdk again I'd contact you and we could work >>>>> together comparing what break in your setup. >>>>> >>>> >>>> Of course! I'll be the beta tester. ;-) >>>> >>>> Christian Ehrhardt >>>>> Software Engineer, Ubuntu Server >>>>> Canonical Ltd >>>>> >>>> >>>> Just for the record, we moved this thread to Ubuntu Server, here is the >>>> previous thread on Ubuntu Devel, so, people on Ubuntu Server will >>>> understand what we're talking about: >>>> >>>> >>>> https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2016-March/016287.html >>>> >>>> Cheers! >>>> Thiago >>>> >>> >>> >> >
-- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
