Re: [ovs-discuss] Issue when using local_ip with VXLAN tunnels in OVS

Tue, 06 Nov 2018 14:43:23 -0800 


On 11/6/2018 8:51 AM, Siva Teja ARETI wrote:

Hi Greg,

Thanks for looking into this.
I have two VMs in my setup each with two interfaces. Trying to setup the VXLAN tunnels across these interfaces which are in different subnets. A docker container is attached to ovs bridge using ovs-docker utility on each VM and doing a ping from one container to another. 

Hi Siva,
In reading through the documentation and looking at your configuration I noticed that when using the local_ip option the remote_ip is not set to flow.  If the local_ip option is specified then remote_ip must 
equal flow.

From the documentation (man ovs-vswitchd.conf.db):

       options : local_ip: optional string
              Optional.  The  tunnel destination IP that received packets must               match. Default is to match all addresses. If specified,  may  be 
              one of:
              ·      An IPv4/IPv6 address (not a DNS name), e.g. 192.168.12.3.
              ·      The  word flow. The tunnel accepts packets sent to any of                      the local IP addresses of  the  system running  OVS.  To                      process  only  packets sent to a specific IP address, the                      flow entries may match on  the  tun_dst  or tun_ipv6_dst                      field.  When  sending  packets to a local_ip=flow tunnel,                      the flow  actions  may  explicitly  set  the tun_src  or                      tun_ipv6_src field to the desired IP address, e.g. with a                      set_field action. However,  while  routing the  tunneled                      packet  out,  the local system may override the specified                      address with the local IP address configured for the out‐ 
                     going system interface.
                     This  option  is  valid  only for tunnels also configured 
                     with the remote_ip=flow option.
Please try using the remote_ip=flow option and then configuring the proper flow and action. 

Thanks,

- Greg



*VM1 details:*

[root@vm1 ~]# ip a
.......
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 
    link/ether 52:54:00:b8:05:be brd ff:ff:ff:ff:ff:ff
    inet 30.30.0.59/24 <http://30.30.0.59/24> brd 30.30.0.255 scope global dynamic eth1 
       valid_lft 3002sec preferred_lft 3002sec
    inet6 fe80::5054:ff:feb8:5be/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 
    link/ether 52:54:00:f0:64:37 brd ff:ff:ff:ff:ff:ff
    inet 20.20.0.183/24 <http://20.20.0.183/24> brd 20.20.0.255 scope global dynamic eth2 
       valid_lft 3248sec preferred_lft 3248sec
    inet6 fe80::5054:ff:fef0:6437/64 scope link
       valid_lft forever preferred_lft forever
.......
[root@vm1 ~]# ovs-vsctl show
ff70c814-d1b0-4018-aee8-8b635187afee
    Bridge "testbr0"
        Port "gre0"
            Interface "gre0"
                type: gre
                options: {local_ip="20.20.0.183", remote_ip="30.30.0.193"}
        Port "testbr0"
            Interface "testbr0"
                type: internal
        Port "2cfb62a9b0f04_l"
            Interface "2cfb62a9b0f04_l"
    ovs_version: "2.9.2"
[root@vm1 ~]# ip rule list
0:      from all lookup local
32765:  from 20.20.0.183 lookup siva
32766:  from all lookup main
32767:  from all lookup default
[root@vm1 ~]# ip route show table siva
default dev eth2 scope link src 20.20.0.183
[root@vm1 ~]# ######################### A docker container is attached to ovs bridge using ovs-docker utility 
[root@vm1 ~]# docker ps
CONTAINER ID        IMAGE            COMMAND  CREATED             STATUS         PORTS  NAMES be4ab434db99        busybox            "sh"                5 days ago          Up 5 days  admiring_euclid [root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` -- ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
    link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000 
    link/ether 22:98:41:0f:e8:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 70.70.0.10/24 <http://70.70.0.10/24> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::2098:41ff:fe0f:e850/64 scope link
       valid_lft forever preferred_lft forever


*VM2 details:*
*
*
[root@vm2 ~]# ip a
........
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 
    link/ether 52:54:00:79:ef:92 brd ff:ff:ff:ff:ff:ff
    inet 30.30.0.193/24 <http://30.30.0.193/24> brd 30.30.0.255 scope global dynamic eth1 
       valid_lft 2406sec preferred_lft 2406sec
    inet6 fe80::5054:ff:fe79:ef92/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 
    link/ether 52:54:00:05:93:7c brd ff:ff:ff:ff:ff:ff
    inet 20.20.0.64/24 <http://20.20.0.64/24> brd 20.20.0.255 scope global dynamic eth2 
       valid_lft 2775sec preferred_lft 2775sec
    inet6 fe80::5054:ff:fe05:937c/64 scope link
       valid_lft forever preferred_lft forever
.......
[root@vm2 ~]# ovs-vsctl show
b85514db-3f29-4f7a-9001-37d70adfca34
    Bridge "testbr0"
        Port "gre0"
            Interface "gre0"
                type: gre
                options: {local_ip="30.30.0.193", remote_ip="20.20.0.183"}
        Port "a0769422cfc04_l"
            Interface "a0769422cfc04_l"
        Port "testbr0"
            Interface "testbr0"
                type: internal
    ovs_version: "2.9.2"
[root@vm2 ~]# ip rule list
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default
[root@vm2 ~]# ######################### A docker container is attached to ovs bridge using ovs-docker utility 
[root@vm2 ~]# docker ps
CONTAINER ID        IMAGE              COMMAND    CREATED             STATUS             PORTS  NAMES 86214f0d99e8 busybox:latest      "sh"           5 days ago Up 5 days          peaceful_snyder [root@vm2 ~]# nsenter -n -t `docker inspect 862 --format={{.State.Pid}}` -- ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
    link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN qlen 1000 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000 
    link/ether ae:ac:14:7a:40:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 70.70.0.20/24 <http://70.70.0.20/24> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::acac:14ff:fe7a:405f/64 scope link
       valid_lft forever preferred_lft forever
With this configuration, if I do a ping from docker container on VM1 to docker container on VM2 it works.
[root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` -- ping 70.70.0.20 
PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=1 ttl=64 time=0.831 ms 64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=2 ttl=64 time=0.933 ms 64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=3 ttl=64 time=0.564 ms 
^C
--- 70.70.0.20 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.564/0.776/0.933/0.155 ms

And the traffic is as expected on VM2.

[root@vm2 ~]# tcpdump -n -i any host 20.20.0.183
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 16:37:32.262553 IP 20.20.0.183 > 30.30.0.193 <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 > 70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq 1, length 64 16:37:32.262835 IP 30.30.0.193 > 20.20.0.183 <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 > 70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 1, length 64 16:37:33.263211 IP 20.20.0.183 > 30.30.0.193 <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 > 70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq 2, length 64 16:37:33.263374 IP 30.30.0.193 > 20.20.0.183 <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 > 70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 2, length 64 16:37:34.264159 IP 20.20.0.183 > 30.30.0.193 <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 > 70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq 3, length 64 16:37:34.264252 IP 30.30.0.193 > 20.20.0.183 <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 > 70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 3, length 64 

But when I change the tunnel type to vxlan, ping fails.

[root@vm1 ~]# ovs-vsctl del-port testbr0 gre0
[root@vm1 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface vxlan0 type=vxlan options:local_ip=20.20.0.183 options:remote_ip=30.30.0.193 options:dst_port=4789 
[root@vm1 ~]# ovs-vsctl show
ff70c814-d1b0-4018-aee8-8b635187afee
    Bridge "testbr0"
        Port "testbr0"
            Interface "testbr0"
                type: internal
        Port "vxlan0"
            Interface "vxlan0"
                type: vxlan
                options: {dst_port="4789", local_ip="20.20.0.183", remote_ip="30.30.0.193"} 
        Port "2cfb62a9b0f04_l"
            Interface "2cfb62a9b0f04_l"
    ovs_version: "2.9.2"

[root@vm2 ~]# ovs-vsctl del-port testbr0 gre0
[root@vm2 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface vxlan0 type=vxlan options:local_ip=30.30.0.193 options:remote_ip=20.20.0.183 options:dst_port=4789 
[root@vm2 ~]# ovs-vsctl show
b85514db-3f29-4f7a-9001-37d70adfca34
    Bridge "testbr0"
        Port "a0769422cfc04_l"
            Interface "a0769422cfc04_l"
        Port "vxlan0"
            Interface "vxlan0"
                type: vxlan
                options: {dst_port="4789", local_ip="30.30.0.193", remote_ip="20.20.0.183"} 
        Port "testbr0"
            Interface "testbr0"
                type: internal
    ovs_version: "2.9.2"

Ping fails with this setup
[root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` -- ping 70.70.0.20 
PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
^C
--- 70.70.0.20 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 4999ms

Expected traffic is not seen on VM2

[root@vm2 ~]# tcpdump -n -i any host 20.20.0.183
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

Kindly let me know if you need more information.

Siva Teja.
On Tue, Nov 6, 2018 at 10:49 AM Gregory Rose <gvrose8...@gmail.com <mailto:gvrose8...@gmail.com>> wrote: 


    On 11/5/2018 6:10 PM, Siva Teja ARETI wrote:

    Hi,

    I am trying to use local_ip option for a VXLAN tunnel using ovs
    but it does not seem to work. The same works when I use GRE
    tunnel. I also found a previous discussion from another user who
    tried the exact same approach. Here is the link to the discussion

    _https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html_

    I am unable to find any working resolution at the end of this
    discussion. Could you please help?


    I looked into that but was never able to set up a configuration
    like the one in that discussion and could
    not repro the bug.

    Please provide some details on your usage, configuration and steps
    to repro and I can look into it.

    Thanks,

    - Greg



    I am using ovs 2.9.2

    [root@localhost ~]# ovs-vsctl --version
    ovs-vsctl (Open vSwitch) 2.9.2
    DB Schema 7.15.1

    Thanks,
    Siva Teja.


    _______________________________________________
    discuss mailing list
    disc...@openvswitch.org <mailto:disc...@openvswitch.org>
    https://mail.openvswitch.org/mailman/listinfo/ovs-discuss





_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Reply via email to