Please ignore my question on starting ovs-vswitchd in 2nd container. I solved that by giving each ovs-vswitchd its own hugepages directory. Then I run into different issue. On my machine, I have 126 SRIOV VFs. Each VF is bounded to DPDK. When starting ovs-vswitchd, it was able to deal with port 0-31. I died when dealing with 32nd port: ...... PMD: eth_ixgbevf_dev_init(): port 31 vendorID=0x8086 deviceID=0x10ed mac.type=ixgbe_mac_82599_vf EAL: PCI device 0000:42:14.0 on NUMA socket 1 EAL: probe driver: 8086:10ed rte_ixgbevf_pmd EAL: PCI memory mapped at 0x7f10e8f00000 EAL: PCI memory mapped at 0x7f10e8f04000 EAL: Error - exiting with code: 1 Cause: Requested device 0000:42:14.0 cannot be used Is there a parameter or code some where I can increase the number of ports that vswitchd can deal with?
John On Wed, Mar 2, 2016 at 4:32 PM, John Wei <[email protected]> wrote: > Hi Mark, > Really appreciate your input. With your suggestion, I was able to pick the > DPDK port be bounded with ovs-vswitchd.This helped me moved further. Now I > realize the issue is that I can't run multiple copies of ovs-vswitchd in > different containers. :) > Since containers is supposed to give you virtual environment similar to > VM, except shared kernel. I was trying to set up OVS+DPDK in container, > with multiple containers. I am using SRIOV VF. Each vswitchd in each > container will be using two dpdk ports. > > With your suggestion, I can assign different dpdk ports to different > vswtichd in different contains. As I experiment further, i found the issue > is that I can't even bring up 2nd vswitchd in 2nd container. > > As soon I tried to run this command in 2nd container: > ovs-vswitchd --dpdk -c 0x2 -n 4 -- unix:$DB_SOCK --pidfile --detach > --log-file > > The ovs-vswitchd in 1st container exit without error message in the > log-file. > > I wonder if there is way to work around this? > > Thanks for your help. > > John > > > > On Wed, Mar 2, 2016 at 7:07 AM, Kavanagh, Mark B < > [email protected]> wrote: > >> Hi John, >> >> If I understand your question correctly, you have multiple NIC ports >> bound to the DPDK driver, and of those, you want to bind a specific port to >> ovs-vswitchd? >> >> It would help me to answer your query more effectively if I had some more >> information on what your setup looks like, and what your desired OVS >> end-state configuration is, but this answer should provide some general >> guidance. >> >> To provide some context, consider how the DPDK port enumeration process >> works, which is (roughly) as follows: >> - as part of the DPDK init process, PCI devices are enumerated >> - enumerated devices are typically ordered from low-to-high by PCI >> Bus:Device:Function >> - for each compatible PCI function detected, DPDK checks if that PCI >> device is bound to a supported DPDK driver >> - if so, that device is assigned a port ID, starting from 0 for the >> lowest PCI ID, and increasing to 1 for the next PCI ID, and so on >> - for more information, consult DPDK documentation and/or the DPDK >> mailing list >> >> In your case, you have (presumably) two PCI functions/NIC ports bound to >> a DPDK driver; on init, these devices are enumerated/indexed as 0 and 1, >> respectively. >> >> Consequently, to add DPDK port 'N' to a bridge, simply run: >> >> 'ovs-vsctl add-port br0 dpdkN -- set Interface dpdkN type=dpdk' >> >> >> Where N = the index of the DPDK device port, as enumerated by DPDK on >> init. >> >> I hope this helps. >> >> Cheers, >> Mark >> >> >> >>URL added: >> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md >> > >> >On Tue, Mar 1, 2016 at 3:22 PM, John Wei <[email protected]> wrote: >> >I am following instructions in this URL to setup ovs to use with DPDK. >> >With the instruction, the first two dpdk ports were always used bu ovs: >> >ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk >> >ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk >> > >> >How to tell ovs-vswitchd to use other dpdk ports? >> > >> >John >> > >
_______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
