Re: [ovs-discuss] Starting OVS w/ DPDK ...
On Tue, Oct 23, 2018 at 11:53:30AM +0200, Andrzej Ostruszka wrote:
> Flavio
>
> Thank you again for trying to help me. Luckily I've solved the problem.
> So here I'll just document it for the benefit of other people with the
> same problem.
Hi Andrzej,
Glad to know that you solved the problem. I am not sure if you're
familiar with git and patch. If you are, perhaps this would be a
great opportunity for you to change the relevant documentation to
add your findings and send us a patch.
If you're not comfortable in doing it, I'd suggest to send a
documentation snippet, then we can try to integrate it.
Thanks,
fbl
>
> On 10/19/2018 08:12 PM, Flavio Leitner wrote:
> > On Mon, Oct 08, 2018 at 11:52:24AM +0200, Andrzej Ostruszka wrote:
> [...]
> >> ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
> >> device '08:00.0' to DPDK. See ovs-vswitchd log for details.
> >> ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".
> >>
> >> and in there the confirmation from DPDK:
> >>
> >> 2018-10-08T09:38:28.234Z|00058|dpdk|ERR|EAL: Cannot find unplugged
> >> device (08:00.0)
> >> 2018-10-08T09:38:28.234Z|00059|netdev_dpdk|WARN|Error attaching device
> >> '08:00.0' to DPDK
> >> 2018-10-08T09:38:28.234Z|00060|netdev|WARN|dpdk-p0: could not set
> >> configuration (Invalid argument)
>
> The key word in the above log was "unplugged". What actually happened
> when I have given PCI address is that OvS tried to attach the driver
> when it already has been attached.
>
> So it looks like upon the startup OvS does a scan and attaches the ports
> (excerpt from the log):
> |INFO|EAL: PCI device :01:00.0 on NUMA socket -1
> |WARN|EAL: Invalid NUMA socket, default to 0
> |INFO|EAL: probe driver: 15b3:1019 net_mlx5
> |INFO|PMD: net_mlx5: PCI information matches, using device "mlx5_0"
> (SR-IOV: false)
> |INFO|PMD: net_mlx5: 1 port(s) detected
>
> So I've taken a look at the code and saw that OvS takes the dpdk-devargs
> and first searches for the port by that name and if not found then it
> tries to attach/hotplug using this value. Thus the solution was to
> change the value of dpdk-devargs to the name of the port (mlx5_0 and
> mlx5_1) instead of PCI address. So with the following config everything
> works just fine - I guess the documentation could be updated a bit though.
>
> # ovs-vsctl show
> 67be421b-d397-4173-b1d5-e7d80f641daa
> Bridge "br0"
> Port "dpdk-p1"
> Interface "dpdk-p1"
> type: dpdk
> options: {dpdk-devargs="mlx5_1"}
> Port "br0"
> Interface "br0"
> type: internal
> Port "dpdk-p0"
> Interface "dpdk-p0"
> type: dpdk
> options: {dpdk-devargs="mlx5_0"}
> ovs_version: "2.10.90"
>
> Best regards
> Andrzej
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
--
Flavio
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Starting OVS w/ DPDK ...
Flavio
Thank you again for trying to help me. Luckily I've solved the problem.
So here I'll just document it for the benefit of other people with the
same problem.
On 10/19/2018 08:12 PM, Flavio Leitner wrote:
> On Mon, Oct 08, 2018 at 11:52:24AM +0200, Andrzej Ostruszka wrote:
[...]
>> ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
>> device '08:00.0' to DPDK. See ovs-vswitchd log for details.
>> ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".
>>
>> and in there the confirmation from DPDK:
>>
>> 2018-10-08T09:38:28.234Z|00058|dpdk|ERR|EAL: Cannot find unplugged
>> device (08:00.0)
>> 2018-10-08T09:38:28.234Z|00059|netdev_dpdk|WARN|Error attaching device
>> '08:00.0' to DPDK
>> 2018-10-08T09:38:28.234Z|00060|netdev|WARN|dpdk-p0: could not set
>> configuration (Invalid argument)
The key word in the above log was "unplugged". What actually happened
when I have given PCI address is that OvS tried to attach the driver
when it already has been attached.
So it looks like upon the startup OvS does a scan and attaches the ports
(excerpt from the log):
|INFO|EAL: PCI device :01:00.0 on NUMA socket -1
|WARN|EAL: Invalid NUMA socket, default to 0
|INFO|EAL: probe driver: 15b3:1019 net_mlx5
|INFO|PMD: net_mlx5: PCI information matches, using device "mlx5_0"
(SR-IOV: false)
|INFO|PMD: net_mlx5: 1 port(s) detected
So I've taken a look at the code and saw that OvS takes the dpdk-devargs
and first searches for the port by that name and if not found then it
tries to attach/hotplug using this value. Thus the solution was to
change the value of dpdk-devargs to the name of the port (mlx5_0 and
mlx5_1) instead of PCI address. So with the following config everything
works just fine - I guess the documentation could be updated a bit though.
# ovs-vsctl show
67be421b-d397-4173-b1d5-e7d80f641daa
Bridge "br0"
Port "dpdk-p1"
Interface "dpdk-p1"
type: dpdk
options: {dpdk-devargs="mlx5_1"}
Port "br0"
Interface "br0"
type: internal
Port "dpdk-p0"
Interface "dpdk-p0"
type: dpdk
options: {dpdk-devargs="mlx5_0"}
ovs_version: "2.10.90"
Best regards
Andrzej
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Starting OVS w/ DPDK ...
Hi,
On Mon, Oct 08, 2018 at 11:52:24AM +0200, Andrzej Ostruszka wrote:
> First of all thank you Flavio for your reply and sorry for late reply
> but I had to switch to something else. Now I resume my ovs ramp up and
> have some additional problems.
>
> My goal is actually to measure performance gain resulting from
> offloading of flow matching to HW. I started with Marvell (even though
> it does not support at the moment RSS+MARK action in DPDK flow API)
> since I thought that if I get it up and running that would be a good
> starting point for adding support for flow matching offload to Marvell
> PMD. However I do have problems even with what seems to be already
> supported.
>
> For the moment I suspend the starting of OVS/DPDK on Marvell PMD and I'm
> trying to bring up the OVS with Mellanox MLX5 PMD. That should be
> supported according to the documentation however I fail to bring it up.
> Maybe there is something very basic that I get wrong so I'd like to
> solicit some help to first start OVS/DPDK on this card (MCX516A-CDAT).
> Once I get this up and running I'll move to Marvell PMDs.
>
> Let me shortly describe my attempt - maybe you'll spot something wrong.
> 1. I compile and install DPDK and test if testpmd has no problems
> (output at the end [1] in order to not clutter this e-mail too much).
>
> 2. Then I configure ovs
>
> ./configure --with-dpdk=$HOME/dpdk-install/usr/local
> --with-linux=/lib/modules/$(uname -r)/build LIBS="-libverbs -lmlx5"
>
> and install it.
>
> 3. I tell OVS to initialize DPDK
>
> # ovs-vsctl get Open_vSwitch . other_config
> {dpdk-init="true"}
>
> 4. I start it (without any bridge)
>
> # ovs-ctl --delete-bridges start
>
> * Starting ovsdb-server
> * system ID not configured, please use --system-id
> * Configuring Open vSwitch system IDs
> Zone 0: name:, IO:0x5e9b6000, len:0x34900,
> virt:0x7f8dd29b6000, socket_id:0, flags:0
> * Starting ovs-vswitchd
> * Enabling remote OVSDB managers
>
> 5. Then I attach to ovs-vswitchd in gdb and set breakpoint in
> rte_eal_hotplug_add
>
> 6. Then I add bridge and first DPDK port
>
> # ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
> # ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
> options:dpdk-devargs=08:00.0
>
> 7. In the debugger I step through until I get to pci_find_device() and
> in there the comparison function used is 'cmp_detached_dev_name()' which
> checks for dev->drive to be NULL and in my case it is not as it points
> to mlx5 driver:
>
> (gdb) p name
> $23 = 0x262c250 "08:00.0"
> (gdb) p *dev
> $24 = {next = {tqe_next = 0x0, tqe_prev = 0x0}, name = 0x25fa5c0 "08:00.0",
> driver = 0xc96b30 , numa_node = 0, devargs = 0x25fa5a0}
> (gdb) n
> 55 return 1;
>
> and as a result it fails to add port:
>
> ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
> device '08:00.0' to DPDK. See ovs-vswitchd log for details.
> ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".
>
> and in there the confirmation from DPDK:
>
> 2018-10-08T09:38:28.234Z|00058|dpdk|ERR|EAL: Cannot find unplugged
> device (08:00.0)
> 2018-10-08T09:38:28.234Z|00059|netdev_dpdk|WARN|Error attaching device
> '08:00.0' to DPDK
> 2018-10-08T09:38:28.234Z|00060|netdev|WARN|dpdk-p0: could not set
> configuration (Invalid argument)
>
> So basically port seems to be attached (to correct driver) but somehow
> detached port is searched for. Seems very much like some config problem
> but I don't know OVS enough to figure it out.
>
> So I'd still appreciate some help in bringing OVS w/ DPDK up.
I've not tried Mellanox myself in a while now, but I found a piece of
documentation that you might have missed:
Documentation/howto/dpdk.rst:
[...]
Some NICs (i.e. Mellanox ConnectX-3) have only one PCI address associated with
multiple ports. Using a PCI device like above won't work. Instead, below usage
is suggested::
$ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0
type=dpdk \
options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55"
$ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1
type=dpdk \
options:dpdk-devargs="class=eth,mac=00:11:22:33:44:56"
[...]
> > BTW the bridge 'br0' also needs to be in the userspace DP.
>
> Ohh, that seems to be in contrast to what this doc:
> http://docs.openvswitch.org/en/latest/howto/dpdk/
> seems to suggest. It tells that the bridge has to be netdev (or maybe I
> don't understand what the 'netdev' datapath type is).
That is correct.
--
Flavio
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Starting OVS w/ DPDK ...
First of all thank you Flavio for your reply and sorry for late reply
but I had to switch to something else. Now I resume my ovs ramp up and
have some additional problems.
My goal is actually to measure performance gain resulting from
offloading of flow matching to HW. I started with Marvell (even though
it does not support at the moment RSS+MARK action in DPDK flow API)
since I thought that if I get it up and running that would be a good
starting point for adding support for flow matching offload to Marvell
PMD. However I do have problems even with what seems to be already
supported.
For the moment I suspend the starting of OVS/DPDK on Marvell PMD and I'm
trying to bring up the OVS with Mellanox MLX5 PMD. That should be
supported according to the documentation however I fail to bring it up.
Maybe there is something very basic that I get wrong so I'd like to
solicit some help to first start OVS/DPDK on this card (MCX516A-CDAT).
Once I get this up and running I'll move to Marvell PMDs.
Let me shortly describe my attempt - maybe you'll spot something wrong.
1. I compile and install DPDK and test if testpmd has no problems
(output at the end [1] in order to not clutter this e-mail too much).
2. Then I configure ovs
./configure --with-dpdk=$HOME/dpdk-install/usr/local
--with-linux=/lib/modules/$(uname -r)/build LIBS="-libverbs -lmlx5"
and install it.
3. I tell OVS to initialize DPDK
# ovs-vsctl get Open_vSwitch . other_config
{dpdk-init="true"}
4. I start it (without any bridge)
# ovs-ctl --delete-bridges start
* Starting ovsdb-server
* system ID not configured, please use --system-id
* Configuring Open vSwitch system IDs
Zone 0: name:, IO:0x5e9b6000, len:0x34900,
virt:0x7f8dd29b6000, socket_id:0, flags:0
* Starting ovs-vswitchd
* Enabling remote OVSDB managers
5. Then I attach to ovs-vswitchd in gdb and set breakpoint in
rte_eal_hotplug_add
6. Then I add bridge and first DPDK port
# ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
# ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk
options:dpdk-devargs=08:00.0
7. In the debugger I step through until I get to pci_find_device() and
in there the comparison function used is 'cmp_detached_dev_name()' which
checks for dev->drive to be NULL and in my case it is not as it points
to mlx5 driver:
(gdb) p name
$23 = 0x262c250 "08:00.0"
(gdb) p *dev
$24 = {next = {tqe_next = 0x0, tqe_prev = 0x0}, name = 0x25fa5c0 "08:00.0",
driver = 0xc96b30 , numa_node = 0, devargs = 0x25fa5a0}
(gdb) n
55 return 1;
and as a result it fails to add port:
ovs-vsctl: Error detected while setting up 'dpdk-p0': Error attaching
device '08:00.0' to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".
and in there the confirmation from DPDK:
2018-10-08T09:38:28.234Z|00058|dpdk|ERR|EAL: Cannot find unplugged
device (08:00.0)
2018-10-08T09:38:28.234Z|00059|netdev_dpdk|WARN|Error attaching device
'08:00.0' to DPDK
2018-10-08T09:38:28.234Z|00060|netdev|WARN|dpdk-p0: could not set
configuration (Invalid argument)
So basically port seems to be attached (to correct driver) but somehow
detached port is searched for. Seems very much like some config problem
but I don't know OVS enough to figure it out.
So I'd still appreciate some help in bringing OVS w/ DPDK up.
Best regards
Andrzej
PS. Below are some comments to your last reply Flavio.
On 17.09.2018 20:25, Flavio Leitner wrote:
> On Thu, Sep 13, 2018 at 04:56:51PM +0200, Andrzej Ostruszka wrote:
[...]
>> modprobe: ERROR: could not insert 'openvswitch': Unknown symbol in
>> module, or unknown parameter (see d
>> mesg)
>
> You can work around that in modprobe.conf, e.g.:
> install openvswitch /bin/true
Thanks I have not thought about that.
[...]
>> # ovs-vsctl show
>> 78e4732f-e2e1-4784-971c-f4d67e163a26
>>
>> Bridge "br0"
>>
>> Port "br0"
>>
>> Interface "br0"
>>
>> type: internal
>>
>> ovs_version: "2.9.2"
>>
>>
>> so it looks like I'm good to go but I get problems when I try to add
>> ports to it.
>
> BTW the bridge 'br0' also needs to be in the userspace DP.
Ohh, that seems to be in contrast to what this doc:
http://docs.openvswitch.org/en/latest/howto/dpdk/
seems to suggest. It tells that the bridge has to be netdev (or maybe I
don't understand what the 'netdev' datapath type is).
[...]
> I'd say to stop trying with OVS and try ``testpmd´´ from DPDK directly and
> see if you can get it working. It is the simplest possible scenario
> where you can tell that at least the bare bones are working.
The testpmd on that board was running without any problems. And the
same is for the case of MLX5 based board as mentioned above. I must be
missing something.
Best regards
Andrzej
[1] Sample output with testing of MLX5 via testpmd (the links are down
since I have not yet plugged anything)
# ./testpmd -w 08:00.0 -w 08:00.1 -- -i
EAL: Detected 4 lcore(s)
Re: [ovs-discuss] Starting OVS w/ DPDK ...
On Thu, Sep 13, 2018 at 04:56:51PM +0200, Andrzej Ostruszka wrote: > First of all, thank you for your reply Flavio. > > On 12.09.2018 19:33, Flavio Leitner wrote: > > On Wed, Sep 12, 2018 at 09:21:18AM +0200, Andrzej Ostruszka wrote: > [...] > >> but when I try to start the vswitchd I'm getting errors: > >> > >> # ovs-ctl --no-ovsdb-server start > > > > I don't know which OVS version is this and it may be because ovs-ctl > > is trying to load the module or because ovs-vswitchd is doing that. > > So, running sh -x ovs-ctl ... might shed a bit of light. > > The version is 2.9.2 (build from git tag v2.9.2 with the previously > mentioned configure line). > > This is the ovs-ctl that is unconditionally trying to load modules. > > start -> start_ovsdb & start_forwarding > start_forwarding -> do_start_forwarding -> insert_mod_if_required Yeah, the initialization scripts doesn't know what is in the db, so it seems convenient to just load the module. > and "if_required" part of that function only skips loading if module is > already present or there is no support in kernel for modules. Below are > some snippets from "sh -x > /usr/local/ovs/share/openvswitch/scripts/ovs-ctl --no-ovsdb-server start" [...] > modprobe: ERROR: could not insert 'openvswitch': Unknown symbol in > module, or unknown parameter (see d > mesg) You can work around that in modprobe.conf, e.g.: install openvswitch /bin/true > I'm not sure why openvswitch module is needed in case of using DPDK > based ports. That might be just the case of documentation not being up > to date (in case it is needed) or missing checks in ovs-ctl (in case it > can be skipped for DPDK based switching). I suspect it is the latter. > Anyway I've decided to reconfigure my kernel to match requirements of > openvswitch module (some config options were missing) and now I have > some progress (I load Marvell's modules, configure huge pages first, > then I start db and set dpd-init to true - as previously): > > # ovs-ctl --no-ovsdb-server start > * Starting ovs-vswitchd > > * Enabling remote OVSDB managers > > # ovs-ctl status > ovsdb-server is running with pid 2928 > > ovs-vswitchd is running with pid 3055 > > # ovs-vsctl show > 78e4732f-e2e1-4784-971c-f4d67e163a26 > > Bridge "br0" > > Port "br0" > > Interface "br0" > > type: internal > > ovs_version: "2.9.2" > > > so it looks like I'm good to go but I get problems when I try to add > ports to it. BTW the bridge 'br0' also needs to be in the userspace DP. > # ovs-vsctl add-port br0 port0 -- set Interface port0 type=dpdk > options:dpdk-devargs=eth_mvpp20,iface=eth0 ofport_request=1 > > [ 7156.926062] pgd = ffc23465a000 > > [ 7156.927215] pgd = ffc23465a000 > > [ 7156.927278] [0008] *pgd=000235346003, *pud=000235346003, > *pmd= > [ 7156.941365] [] *pgd=000235346003, *pud=000235346003, > *pmd= > [ 7161.622915] pgd = ffc1ddd54000 > > [ 7161.622938] [0008] *pgd=0001ddd55003, *pud=0001ddd55003, > *pmd= > > and I don't get command prompt again unless I interrupt with ^C. > I have no idea what ofport_request=1 is for - I'm just copying from the > docs (I assume this just gives id to port). This type of log is The ofport_request tells the switch that you want a specific OpenFlow port ID for that port. If you don't provide it, then the vswitch will get the next available to you. > periodic - ovs-vswitchd seems to be crashing and is being periodically > restarted. In dmesg I see periodic entries as in [1] and in > var/run/openvswitch I get ovs-vswitchd.PID.ctl sockets being added. > > So I infer that for some reason vswitchd is crashing and I'd like to > investigate that. Any tips on how could I proceed with it? Look at ovs-vswitchd.log and possibly the core dump file. OVS can start the monitoring threads, then if the vswitchd crashes, it will try to bring it up again. There are some safe guards there to avoid loops. > Best regards > Andrzej > > [1] example of dmesg entry: > --8< I'd say to stop trying with OVS and try ``testpmd´´ from DPDK directly and see if you can get it working. It is the simplest possible scenario where you can tell that at least the bare bones are working. > [10262.208121] CMA: total alloc 314, total free: 314 (garbage 1) > [10271.717382] CMA buffer allocated: size = 41943040 Bytes, kvaddr = > ffc1de5f6000, paddr = 0xb050 > > [10271.717458] CMA buffer remapped: vm_start=0x7f3c80, size = > 41943040 bytes, paddr = 0xb050 > [10271.893887] pmd8[6947]: unhandled level 2 translation fault (11) at > 0x, esr 0x9206 > [10271.893893] pgd = ffc1de506000 > [10271.893957] [6936]: unhandled level 2 translation fault (11) at > 0x0008, esr 0x9206 > [10271.893958] pgd = ffc1de506000 > [10271.893962] [0008] *pgd=0001dde36003, *pud=0001dde36003, >
Re: [ovs-discuss] Starting OVS w/ DPDK ...
On Wed, Sep 12, 2018 at 09:21:18AM +0200, Andrzej Ostruszka wrote:
> Hello all
>
> I'm new to OVS so please have some patience with me :)
>
> What I'm trying to do is to start OVS with DPDK based ports.
> The actual setup is a board with Marvell's Armada 8040 chip.
>
> I don't have any problems with starting DPDK based apps, it goes more or
> less like:
> - load required kernel modules (Marvell ones, specific for this chip)
> - start DPDK app and pass to EAL options to create correct vdev:
> ./testpmd ... --vdev=eth_mvpp2,iface=eth0 ...
>
> So my understanding is that I don't need any OVS specific kernel modules
> in order to use these ports via DPDK in OVS. So I build OVS with option
> "--with-dpdk" but without "--with-linux" (as intro/install/dpdk.rst
> seems to suggest) [1]. Then I start the db:
>
> # ovs-ctl --no-ovs-vswitchd start
>
> * Starting ovsdb-server
> * system ID not configured, please use --system-id
> * Configuring Open vSwitch system IDs
>
> and indicate that I want to initialize DPDK:
>
> # ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true
> # ovs-vsctl get Open_vSwitch . other_config:dpdk-init
> "true"
>
> but when I try to start the vswitchd I'm getting errors:
>
> # ovs-ctl --no-ovsdb-server start
I don't know which OVS version is this and it may be because ovs-ctl
is trying to load the module or because ovs-vswitchd is doing that.
So, running sh -x ovs-ctl ... might shed a bit of light.
fbl
>
>
> modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not
> open moddep file
> '/lib/modules/4.4.52-armada-17.10.4-g2dfc733/modules.dep.bin'
> modprobe: FATAL: Module openvswitch not found in directory
> /lib/modules/4.4.52-armada-17.10.4-g2dfc733
> * Inserting openvswitch module
> rmmod: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not
> open builtin file
> '/lib/modules/4.4.52-armada-17.10.4-g2dfc733/modules.builtin.bin'
> rmmod: ERROR: ../libkmod/libkmod-module.c:1921 kmod_module_get_holders()
> could not open '/sys/module/bridge/holders': No such file or directory
> rmmod: ERROR: Module bridge is in use
> * removing bridge module
>
> and obviously vswitchd is not running:
> # ovs-ctl status
> ovsdb-server is running with pid 7566
> ovs-vswitchd is not running
>
> Why there are attempts to load some kernel modules? How should I start
> vswitchd with DPDK based ports? My expectation was that
> "dpdk-init=true" will indicate to ovs to not use any kernel modules and
> that I will be able later to add ports just like:
>
> ovs-vsctl add-port br0 mrvl0 -- set Interface mrvl0 \
> type=dpdk options:dpdk-devargs=eth_mvpp20,iface=eth0
> ovs-vsctl add-port br0 mrvl2 -- set Interface mrvl2 \
> type=dpdk options:dpdk-devargs=eth_mvpp21,iface=eth2
>
> I'd appreciate some tip/insight into what I am doing wrong and/or how
> can I debug this in order to solve it.
>
> Best regards
> Andrzej
>
> [1] Actually I'm cross compiling ovs, the full configure line is:
> ./configure CC=${CROSS}gcc --prefix=/usr/local/ovs
> --with-dpdk=${ROOTDIR}/marvell-dpdk/arm64-armv8a-linuxapp-gcc
> --host=aarch64-linux-gnu --disable-libcapng
> LDFLAGS="-L${ROOTDIR}/install/usr/local/lib -lmusdk"
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
--
Flavio
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Starting OVS w/ DPDK ...
Hello all
I'm new to OVS so please have some patience with me :)
What I'm trying to do is to start OVS with DPDK based ports.
The actual setup is a board with Marvell's Armada 8040 chip.
I don't have any problems with starting DPDK based apps, it goes more or
less like:
- load required kernel modules (Marvell ones, specific for this chip)
- start DPDK app and pass to EAL options to create correct vdev:
./testpmd ... --vdev=eth_mvpp2,iface=eth0 ...
So my understanding is that I don't need any OVS specific kernel modules
in order to use these ports via DPDK in OVS. So I build OVS with option
"--with-dpdk" but without "--with-linux" (as intro/install/dpdk.rst
seems to suggest) [1]. Then I start the db:
# ovs-ctl --no-ovs-vswitchd start
* Starting ovsdb-server
* system ID not configured, please use --system-id
* Configuring Open vSwitch system IDs
and indicate that I want to initialize DPDK:
# ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true
# ovs-vsctl get Open_vSwitch . other_config:dpdk-init
"true"
but when I try to start the vswitchd I'm getting errors:
# ovs-ctl --no-ovsdb-server start
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not
open moddep file
'/lib/modules/4.4.52-armada-17.10.4-g2dfc733/modules.dep.bin'
modprobe: FATAL: Module openvswitch not found in directory
/lib/modules/4.4.52-armada-17.10.4-g2dfc733
* Inserting openvswitch module
rmmod: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not
open builtin file
'/lib/modules/4.4.52-armada-17.10.4-g2dfc733/modules.builtin.bin'
rmmod: ERROR: ../libkmod/libkmod-module.c:1921 kmod_module_get_holders()
could not open '/sys/module/bridge/holders': No such file or directory
rmmod: ERROR: Module bridge is in use
* removing bridge module
and obviously vswitchd is not running:
# ovs-ctl status
ovsdb-server is running with pid 7566
ovs-vswitchd is not running
Why there are attempts to load some kernel modules? How should I start
vswitchd with DPDK based ports? My expectation was that
"dpdk-init=true" will indicate to ovs to not use any kernel modules and
that I will be able later to add ports just like:
ovs-vsctl add-port br0 mrvl0 -- set Interface mrvl0 \
type=dpdk options:dpdk-devargs=eth_mvpp20,iface=eth0
ovs-vsctl add-port br0 mrvl2 -- set Interface mrvl2 \
type=dpdk options:dpdk-devargs=eth_mvpp21,iface=eth2
I'd appreciate some tip/insight into what I am doing wrong and/or how
can I debug this in order to solve it.
Best regards
Andrzej
[1] Actually I'm cross compiling ovs, the full configure line is:
./configure CC=${CROSS}gcc --prefix=/usr/local/ovs
--with-dpdk=${ROOTDIR}/marvell-dpdk/arm64-armv8a-linuxapp-gcc
--host=aarch64-linux-gnu --disable-libcapng
LDFLAGS="-L${ROOTDIR}/install/usr/local/lib -lmusdk"
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
