Yes, it is still visible.
root@node-6:~# ovs-vsctl list interface eth0 #without VF
_uuid : 16d4d75a-baf2-4613-baa5-c9a77d8720e2
admin_state : up
bfd : {}
bfd_status : {}
cfm_fault : []
cfm_fault_status : []
cfm_flap_count : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : full
external_ids : {}
ifindex : 2
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : 16
link_speed : 10000000000
link_state : up
mac : []
mac_in_use : "38:ea:a7:35:dd:60"
mtu : 1500
name : "eth0"
ofport : 1
ofport_request : []
options : {}
other_config : {}
statistics : {collisions=0, rx_bytes=1160109, rx_crc_err=0,
rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=7238,
tx_bytes=1799066, tx_dropped=0, tx_errors=0, tx_packets=9634}
status : {driver_name=ixgbe, driver_version="3.19.1-k",
firmware_version="0x80000684"}
type : ""
root@node-6:~# ovs-vsctl list interface eth0 #with 1 VF enabled
_uuid : 16d4d75a-baf2-4613-baa5-c9a77d8720e2
admin_state : up
bfd : {}
bfd_status : {}
cfm_fault : []
cfm_fault_status : []
cfm_flap_count : []
cfm_health : []
cfm_mpid : []
cfm_remote_mpids : []
cfm_remote_opstate : []
duplex : full
external_ids : {}
ifindex : 2
ingress_policing_burst: 0
ingress_policing_rate: 0
lacp_current : []
link_resets : 16
link_speed : 10000000000
link_state : up
mac : []
mac_in_use : "38:ea:a7:35:dd:60"
mtu : 1500
name : "eth0"
ofport : 1
ofport_request : []
options : {}
other_config : {}
statistics : {collisions=0, rx_bytes=1729, rx_crc_err=0,
rx_dropped=0, rx_errors=0, rx_frame_err=0, rx_over_err=0, rx_packets=9,
tx_bytes=2115, tx_dropped=0, tx_errors=0, tx_packets=25}
status : {driver_name=ixgbe, driver_version="3.19.1-k",
firmware_version="0x80000684"}
type : ""
ovs-vsctl show (eth0/mgmt related part):
Bridge "br-eth0"
Port "br-eth0--br-mgmt"
tag: 102
Interface "br-eth0--br-mgmt"
type: patch
options: {peer="br-mgmt--br-eth0"}
Port "br-eth0"
Interface "br-eth0"
type: internal
Port "br-eth0--br-prv"
Interface "br-eth0--br-prv"
type: patch
options: {peer="br-prv--br-eth0"}
Port "br-eth0--br-ex"
tag: 104
Interface "br-eth0--br-ex"
type: patch
options: {peer="br-ex--br-eth0"}
Port "br-eth0--br-storage"
tag: 105
Interface "br-eth0--br-storage"
type: patch
options: {peer="br-storage--br-eth0"}
Port "eth0"
Interface "eth0"
Bridge br-mgmt
Port br-mgmt
Interface br-mgmt
type: internal
Port "br-mgmt--br-eth0"
Interface "br-mgmt--br-eth0"
type: patch
options: {peer="br-eth0--br-mgmt"}
Not sure what you mean by inserting a static flow to a desired port, but
I've tried to do this earlier, though without any success:
ovs-ofctl add-flow br-eth0 in_port=4,actions=output:6 (4 is br-mgmt, 6 is
eth0)
tcpdumps, while trying to ping other host with VF enabled:
root@node-6:~# tcpdump -i br-mgmt
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-mgmt, link-type EN10MB (Ethernet), capture size 65535 bytes
18:04:06.545409 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:06.549458 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:07.545411 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:07.549437 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:08.567610 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:09.545788 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:09.565425 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:10.545406 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:10.565428 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:11.545405 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:11.582662 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:12.545460 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:04:12.581454 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:04:13.545440 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
(and so on)
root@node-6:~# tcpdump -i br-eth0
tcpdump: WARNING: br-eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:06:29.301447 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:30.319659 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:31.317444 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:32.317441 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:33.335671 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:34.333447 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:35.333444 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:36.350617 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:36.598210 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:37.349453 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:37.597440 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:38.349456 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:38.597411 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:39.366663 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:39.597453 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:40.365439 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:40.597447 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:41.365445 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:06:41.597410 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:06:42.382659 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
(and so on)
root@node-6:~# tcpdump -i eth0
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:07:45.637408 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:07:45.719618 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:46.637412 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:07:46.717445 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:47.098968 LACPv1, length 110
18:07:47.637450 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:07:47.717449 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:47.830263 LLDP, name HP, length 313
18:07:48.637411 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:07:48.735607 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:49.637406 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
18:07:49.733454 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:50.733445 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:51.751608 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:52.749456 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:53.749457 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:54.767539 ARP, Request who-has 192.168.0.2 tell node-6.domain.tld,
length 28
18:07:55.614723 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 28
(and so on)
meanwhile on node-1 (that has 192.168.0.2 and 192.168.0.3):
root@node-1:~# tcpdump -i br-mgmt
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-mgmt, link-type EN10MB (Ethernet), capture size 65535 bytes
18:11:57.479249 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 46
18:11:57.479285 ARP, Reply node-1.domain.tld is-at 96:e1:e9:7d:53:45 (oui
Unknown), length 28
18:11:58.479204 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 46
18:11:58.479245 ARP, Reply node-1.domain.tld is-at 96:e1:e9:7d:53:45 (oui
Unknown), length 28
18:11:59.497440 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 46
18:11:59.497473 ARP, Reply node-1.domain.tld is-at 96:e1:e9:7d:53:45 (oui
Unknown), length 28
18:12:00.495222 ARP, Request who-has node-1.domain.tld tell
node-6.domain.tld, length 46
18:12:00.495257 ARP, Reply node-1.domain.tld is-at 96:e1:e9:7d:53:45 (oui
Unknown), length 28
Seems like the reply doesn't make it to eth0 on node-6. I've tried running
tcpdump on eth6 (VF), but no packages are being captured.
On Wed, Nov 26, 2014 at 8:04 PM, Andrey Korolyov <[email protected]> wrote:
> On Wed, Nov 26, 2014 at 6:54 PM, Dmitry Nikishov <[email protected]>
> wrote:
> > I've just tried installing ovs-switch 2.3.0 and ovs-common 2.3.0, so
> > all the ovs packages versions are aligned. That didn't help: as long
> > as VFs are disabled, it works fine, but not when they are enabled.
> >
> > On Wed, Nov 26, 2014 at 4:27 PM, Andrey Korolyov <[email protected]> wrote:
> >> On Wed, Nov 26, 2014 at 3:17 PM, Dmitry Nikishov <
> [email protected]> wrote:
> >>> Hello all.
> >>>
> >>> I'm trying to get OVS working with Intel 82599 and SR-IOV.
> >>>
> >>> Basically what I currently have is an Ubuntu 12.04 with custom kernel
> >>> (3.14.23) and Intel 82599 NIC (eth0) and OVS 1.10 installed. There is
> >>> also openvswitch-datapath-dkms 2.3.0 which was built for 3.14.23.
> >>>
> >>> There is a br-eth0 bridge in OVS, which is connected to eth0
> >>> interface. Bridge br-mgmt is connected to br-eth0 and is using VLAN
> >>> tag 102.
> >>>
> >>> Network connectivity via br-mgmt -> br-eth0 -> eth0 chain work fine as
> >>> long as 82599 Virtual Functions (VFs) are disabled. Once I enable at
> >>> least 1 VF (echo 1 > /sys/bus/pci/devices/0000:04:00.0/sriov_numvfs),
> >>> server looses connectivity via this chain. If I try to set up VLAN 102
> >>> via ip link directly on eth0, it works.
> >>>
> >>> With the help of tcpdump I've found out that packets don't go any
> >>> further than br-eth0.
> >>>
> >>> Software versions:
> >>> Ubuntu 12.04 with custom kernel 3.14.23
> >>> openvswitch-switch 1.10.1+git20130823-0ubuntu3~cloud0
> >>> openvswitch-datapath-dkms 2.3.0
> >>>
> >>> On Ubuntu 12.04 with 3.11.8-generic, same openvswitch-switch and
> >>> openvswitch-datapath-dkms-lts-saucy there is no such issue.
> >>>
> >>> Any suggestions? Thanks.
> >>> _______________________________________________
> >>> discuss mailing list
> >>> [email protected]
> >>> http://openvswitch.org/mailman/listinfo/discuss
> >>
> >> May be it is better to align dp and vswitch versions first, or try
> >> against -saucy dp with same 3.14, if it is new enough to be built
> >> against it.
>
> As a VF initialization resulting in an effective interface
> reappearance with probably different udev name, did you checked that
> the interface with same name is still visible via netlink, sorry for a
> silly question? If so, try to re-add the interface first after VF
> initialization, if it will not help, please post 'ovs-vsctl list
> Interface eth0' output. Any hints for traffic flow, like inserting
> static flow with desired port# as a destination and looking into
> appropriate tcpdump output, if arp propagation is not likely working,
> are appreciated as well.
>
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
