Hello Christian! Lets make a checklist...
1- Packages "dpdk and openvswitch-switch-dpdk", installed; 2- /etc/default/grub have: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash default_hugepagesz=1GB hugepagesz=1G hugepages=4" GRUB_CMDLINE_LINUX="console=ttyS0,115200n8 BOOTIF=01-b0-83-fe-d7-12-fc" Linux CMD Line: Command line: BOOT_IMAGE=/vmlinuz-4.4.0-13-generic root=/dev/mapper/xenial-1--vg-root ro console=ttyS0,115200n8 BOOTIF=01-b0-83-fe-d7-12-fc quiet splash default_hugepagesz=1GB hugepagesz=1G hugepages=4 vt.handoff=7 3- /etc/dpdk/interfaces have: pci 0000:01:00.0 uio_pci_generic pci 0000:01:00.1 uio_pci_generic 4- /etc/default/openvswitch-switch have: DPDK_OPTS='--dpdk -c 0x1 -n 4 --socket-mem 1024,0' 5- /etc/alternatives/ovs-vswitchd: symbolic link to /usr/lib/openvswitch-switch-dpdk/ovs-vswitchd-dpdk 6- My CPU supports 1G Hugepages (it have the flag: pdpe1gb) 7- I am running most recent Xenial (4.4.0-13-generic), fully upgraded with "apt update ; apt full-upgrade ; reboot" 8- Huge pages on /proc/meminfo # grep Huge /proc/meminfo AnonHugePages: 26624 kB HugePages_Total: 2 HugePages_Free: 1 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 1048576 kB 9- OVS process is running with DPDK options: --- ps xfa | grep dpdk 2634 ? S<s 0:00 ovs-vswit... --dpdk -c 0x1 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor 2635 ? S<Ll 0:00 \_ ovs-vswitchd --dpdk -c 0x1 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor --- 10- DPDK-Compatible Driver activated --- dpdk_nic_bind --status Network devices using DPDK-compatible driver ============================================ 0000:01:00.0 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=uio_pci_generic unused=bnx2x 0000:01:00.1 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=uio_pci_generic unused=bnx2x --- 11- OVS log have no "port 0" / "port 1" # service openvswitch-switch restart # journalctl | grep -i ovs: https://paste.ubuntu.com/15415651/ Also, I'm not seeing any "PMD" lines on my OVS logs! NOTE: after running: "service openvswitch-switch restart", it still have dpdk options (step 9 above still valid) 12- The, OVS commands also fails... # ovs-vsctl show b70c1e0a-20d7-4bdb-98db-467330b72d07 ovs_version: "2.5.0" # ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev # ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk ovs-vsctl: Error detected while setting up 'dpdk0'. See ovs-vswitchd log for details. --- I just don't know what I am missing, I already checked the configurations dozens of times... At least, I'm not alone seeing this problems, since there are more users on Intel's forum with same issue... Thank you for your help! I'm going to IRC now. Best, Thiago On 18 March 2016 at 10:22, Christian Ehrhardt < christian.ehrha...@canonical.com> wrote: > Hi Martin, > didn't find you on IRC today. > You might try to catch me later or on Monday if you like for faster checks > and comparison. > > I once more tried the ovs-dpdk setup. > While I'm blocked later on on other things I clearly can pass the stage > you are hanging on. > > I played a bit around and could cause the error you had by: > 1. not having any card on a dpdk compatible driver > 2. not enabling dpdk in ovs at all > > Could you please check your journalctl when starting openvswitch - it > should have a section like the following per card (ovs-)dpdk initializes: > > ovs-ctl[2155]: PMD: eth_ixgbe_dev_init(): MAC: 4, PHY: 3 > ovs-ctl[2155]: PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 > deviceID=0x1528 > horsea ovs-ctl[2155]: EAL: PCI device 0000:04:00.1 on NUMA socket 0 > horsea ovs-ctl[2155]: EAL: probe driver: 8086:1528 rte_ixgbe_pmd > horsea ovs-ctl[2155]: EAL: PCI memory mapped at 0x7f5d40204000 > horsea ovs-ctl[2155]: EAL: PCI memory mapped at 0x7f5d40404000 > horsea ovs-vswitchd[2189]: PMD: eth_ixgbe_dev_init(): MAC: 4, PHY: 3 > horsea ovs-vswitchd[2189]: PMD: eth_ixgbe_dev_init(): port 0 > vendorID=0x8086 deviceID=0x1528 > > You see "port 0" that will be what is "dpdk0" later on. > > When I e.g. have two devices in ugb_pci_generic and start OVS I get one > for port 0 and one for port 1. > Then I can later on use dpdk0 and dpdk1, but not e.g. dpdk2 > > > > Christian Ehrhardt > Software Engineer, Ubuntu Server > Canonical Ltd > > On Fri, Mar 11, 2016 at 10:17 PM, Martinx - ジェームズ < > thiagocmarti...@gmail.com> wrote: > >> Hey Christian, >> >> Nice job on fixing many DPDK package bugs! :-D >> >> Replying inline below... >> >> On 9 March 2016 at 04:17, Christian Ehrhardt < >> christian.ehrha...@canonical.com> wrote: >> >>> Thanks Jörg for making Martin and me aware. >>> It is kind of a close call, as much of that are new features, but you >>> are right - so I'll be moving this to the Ubuntu-sever mailing list. >>> >>> @Martin: >>> tl;dr - I suggest the following: >>> - you let me know if anything more comes back from the Intel forum post >>> >> >> Absolutely, I'll... >> >> >>> - short Term I'd ask you to reach out on IRC to me on #ubuntu-server - >>> my nick is "cpaelzer" (maybe we find something in an interactive session) >>> >> >> Sounds perfect! I'm there now, "tmartins" nick... >> >> >>> - somewhen in the next weeks I'll be back at openvswitch-dpdk. I'd >>> invite you to a IRC session or a hangout then (let me know what you would >>> prefer and I contact you once I'm ready) >>> >> >> That would be awesome! >> >> >>> >>> >>> Details: >>> The vfio-pci fix (and several others) are just waiting for the upload to >>> happen, if I don't run into any unexpected running blockers that should be >>> today. >>> >> >> Verified, it is working now. >> >> >>> Please let me know if you run into anything else with DPDK as I want to >>> clear as much as possible before release. >>> >> >> After latest release on Xenial, it looks very good! Both UIO and VFIO >> works without effort. >> >> I can easily switch between UIO and VFIO, that DPDK shows the NICs being >> used as "DPDK-Compatible drivers". >> >> Only OpenvSwitch+DPDK doesn't see any "dpdk0 | dpdk1" devices but, I know >> you guys are working into this... >> >> >>> Please feel free to just directly contact me about it - I'd suggest IRC >>> if that work for you. >>> We can move things to mailing lists whenever needed/applicable then. >>> >> >> Sounds great! >> >> >>> >>> Since last week I took a step back and work on more "low level" dpdk >>> testing and issues than the full setup with openvswitch-dpdk. >>> That helps the overall quality, but means my environment is currently >>> not useful to reproduce your case without too much reconfiguration. >>> >> >> That sounds like a good plan, I totally understand that... =) >> >> >>> >>> I highly appreciate to have someone in the Community working with me on >>> it. >>> >> >> You can count on me! I'm a big fan of Ubuntu... >> >> I met Mark in Vancouver on OpenStack summit, he is a great guy and it is >> easier now to see why Ubuntu is so cool! A lot of amazing people working >> here! ^_^ >> >> So I wanted to ask if you could be patient for a few days, once my Env is >>> working for openvswitch-dpdk again I'd contact you and we could work >>> together comparing what break in your setup. >>> >> >> Of course! I'll be the beta tester. ;-) >> >> Christian Ehrhardt >>> Software Engineer, Ubuntu Server >>> Canonical Ltd >>> >> >> Just for the record, we moved this thread to Ubuntu Server, here is the >> previous thread on Ubuntu Devel, so, people on Ubuntu Server will >> understand what we're talking about: >> >> >> https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2016-March/016287.html >> >> Cheers! >> Thiago >> > >
-- ubuntu-server mailing list ubuntu-server@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
