On 11/19/2018 7:50 AM, Siva Teja ARETI wrote:


On Fri, Nov 16, 2018 at 4:52 PM Gregory Rose <gvrose8...@gmail.com <mailto:gvrose8...@gmail.com>> wrote:

    On 11/6/2018 8:51 AM, Siva Teja ARETI wrote:
    Hi Greg,

    Thanks for looking into this.

    I have two VMs in my setup each with two interfaces. Trying to
    setup the VXLAN tunnels across these interfaces which are in
    different subnets. A docker container is attached to ovs bridge
    using ovs-docker utility on each VM and doing a ping from one
    container to another.

    *VM1 details:*

    [root@vm1 ~]# ip a
    .......
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    pfifo_fast state UP qlen 1000
        link/ether 52:54:00:b8:05:be brd ff:ff:ff:ff:ff:ff
        inet 30.30.0.59/24 <http://30.30.0.59/24> brd 30.30.0.255
    scope global dynamic eth1
     valid_lft 3002sec preferred_lft 3002sec
        inet6 fe80::5054:ff:feb8:5be/64 scope link
     valid_lft forever preferred_lft forever
    4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    pfifo_fast state UP qlen 1000
        link/ether 52:54:00:f0:64:37 brd ff:ff:ff:ff:ff:ff
        inet 20.20.0.183/24 <http://20.20.0.183/24> brd 20.20.0.255
    scope global dynamic eth2
     valid_lft 3248sec preferred_lft 3248sec
        inet6 fe80::5054:ff:fef0:6437/64 scope link
     valid_lft forever preferred_lft forever
    .......

    Hi Siva,

    I have a question.  Are you able to ping between the two
    interfaces on VM1 with this command?:

    # ping 20.20.0.183 -I eth1

    thanks,

    - Greg

Hi Greg,

Sorry for the late reply.

Yes, I am able to ping between two interfaces.

[root@localhost ~]# ovs-appctl dpif/show
system@ovs-system: hit:2799 missed:198775
        testbr0:
                a0769422cfc04_l 2/3: (system)
                testbr0 65534/1: (internal)
                vxlan0 10/2: (vxlan: local_ip=30.30.0.193, remote_ip=20.20.0.183)
[root@localhost ~]# ping 20.20.0.183 -I 30.30.0.193
PING 20.20.0.183 (20.20.0.183) from 30.30.0.193 : 56(84) bytes of data.
64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=1 ttl=64 time=0.470 ms 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=2 ttl=64 time=0.657 ms 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=3 ttl=64 time=0.685 ms 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=4 ttl=64 time=0.721 ms 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=5 ttl=64 time=0.630 ms 64 bytes from 20.20.0.183 <http://20.20.0.183>: icmp_seq=6 ttl=64 time=0.629 ms
^C

Well that's probably where my setup isn't configured right.  What is the output of 'ip route' on that system?

Thanks,

- Greg

--- 20.20.0.183 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5000ms
rtt min/avg/max/mdev = 0.470/0.632/0.721/0.079 ms
[root@localhost ~]#

 Siva Teja.

    [root@vm1 ~]# ovs-vsctl show
    ff70c814-d1b0-4018-aee8-8b635187afee
        Bridge "testbr0"
            Port "gre0"
    Interface "gre0"
    type: gre
    options: {local_ip="20.20.0.183", remote_ip="30.30.0.193"}
            Port "testbr0"
    Interface "testbr0"
    type: internal
            Port "2cfb62a9b0f04_l"
    Interface "2cfb62a9b0f04_l"
    ovs_version: "2.9.2"
    [root@vm1 ~]# ip rule list
    0:      from all lookup local
    32765:  from 20.20.0.183 lookup siva
    32766:  from all lookup main
    32767:  from all lookup default
    [root@vm1 ~]# ip route show table siva
    default dev eth2 scope link src 20.20.0.183
    [root@vm1 ~]# ######################### A docker container is
    attached to ovs bridge using ovs-docker utility
    [root@vm1 ~]# docker ps
    CONTAINER ID   IMAGE  COMMAND  CREATED  STATUS PORTS  NAMES
    be4ab434db99   busybox  "sh" 5 days ago Up 5 days  admiring_euclid
    [root@vm1 ~]# nsenter -n -t `docker inspect be4
    --format={{.State.Pid}}` -- ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
    UNKNOWN qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host
           valid_lft forever preferred_lft forever
    2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
        link/gre 0.0.0.0 brd 0.0.0.0
    3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state
    DOWN qlen 1000
        link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
    9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    noqueue state UP qlen 1000
        link/ether 22:98:41:0f:e8:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0
        inet 70.70.0.10/24 <http://70.70.0.10/24> scope global eth0
           valid_lft forever preferred_lft forever
        inet6 fe80::2098:41ff:fe0f:e850/64 scope link
           valid_lft forever preferred_lft forever


    *VM2 details:*
    *
    *
    [root@vm2 ~]# ip a
    ........
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    pfifo_fast state UP qlen 1000
        link/ether 52:54:00:79:ef:92 brd ff:ff:ff:ff:ff:ff
        inet 30.30.0.193/24 <http://30.30.0.193/24> brd 30.30.0.255
    scope global dynamic eth1
     valid_lft 2406sec preferred_lft 2406sec
        inet6 fe80::5054:ff:fe79:ef92/64 scope link
     valid_lft forever preferred_lft forever
    4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    pfifo_fast state UP qlen 1000
        link/ether 52:54:00:05:93:7c brd ff:ff:ff:ff:ff:ff
        inet 20.20.0.64/24 <http://20.20.0.64/24> brd 20.20.0.255
    scope global dynamic eth2
     valid_lft 2775sec preferred_lft 2775sec
        inet6 fe80::5054:ff:fe05:937c/64 scope link
     valid_lft forever preferred_lft forever
    .......
    [root@vm2 ~]# ovs-vsctl show
    b85514db-3f29-4f7a-9001-37d70adfca34
        Bridge "testbr0"
            Port "gre0"
    Interface "gre0"
    type: gre
    options: {local_ip="30.30.0.193", remote_ip="20.20.0.183"}
            Port "a0769422cfc04_l"
    Interface "a0769422cfc04_l"
            Port "testbr0"
    Interface "testbr0"
    type: internal
    ovs_version: "2.9.2"
    [root@vm2 ~]# ip rule list
    0:      from all lookup local
    32766:  from all lookup main
    32767:  from all lookup default
    [root@vm2 ~]# ######################### A docker container is
    attached to ovs bridge using ovs-docker utility
    [root@vm2 ~]# docker ps
    CONTAINER ID     IMAGE      COMMAND        CREATED        
     STATUS           PORTS            NAMES
    86214f0d99e8 busybox:latest   "sh"     5 days ago       Up 5 days
     peaceful_snyder
    [root@vm2 ~]# nsenter -n -t `docker inspect 862
    --format={{.State.Pid}}` -- ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state
    UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
     valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host
     valid_lft forever preferred_lft forever
    2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1
        link/gre 0.0.0.0 brd 0.0.0.0
    3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state
    DOWN qlen 1000
        link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
    9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
    noqueue state UP qlen 1000
        link/ether ae:ac:14:7a:40:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0
        inet 70.70.0.20/24 <http://70.70.0.20/24> scope global eth0
     valid_lft forever preferred_lft forever
        inet6 fe80::acac:14ff:fe7a:405f/64 scope link
     valid_lft forever preferred_lft forever

    With this configuration, if I do a ping from docker container on
    VM1 to docker container on VM2 it works.

    [root@vm1 ~]# nsenter -n -t `docker inspect be4
    --format={{.State.Pid}}` -- ping 70.70.0.20
    PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
    64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=1 ttl=64
    time=0.831 ms
    64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=2 ttl=64
    time=0.933 ms
    64 bytes from 70.70.0.20 <http://70.70.0.20>: icmp_seq=3 ttl=64
    time=0.564 ms
    ^C
    --- 70.70.0.20 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2001ms
    rtt min/avg/max/mdev = 0.564/0.776/0.933/0.155 ms

    And the traffic is as expected on VM2.

    [root@vm2 ~]# tcpdump -n -i any host 20.20.0.183
    tcpdump: verbose output suppressed, use -v or -vv for full
    protocol decode
    listening on any, link-type LINUX_SLL (Linux cooked), capture
    size 262144 bytes
    16:37:32.262553 IP 20.20.0.183 > 30.30.0.193
    <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
    70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
    1, length 64
    16:37:32.262835 IP 30.30.0.193 > 20.20.0.183
    <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
    70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 1,
    length 64
    16:37:33.263211 IP 20.20.0.183 > 30.30.0.193
    <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
    70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
    2, length 64
    16:37:33.263374 IP 30.30.0.193 > 20.20.0.183
    <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
    70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 2,
    length 64
    16:37:34.264159 IP 20.20.0.183 > 30.30.0.193
    <http://30.30.0.193>: GREv0, length 102: IP 70.70.0.10 >
    70.70.0.20 <http://70.70.0.20>: ICMP echo request, id 28158, seq
    3, length 64
    16:37:34.264252 IP 30.30.0.193 > 20.20.0.183
    <http://20.20.0.183>: GREv0, length 102: IP 70.70.0.20 >
    70.70.0.10 <http://70.70.0.10>: ICMP echo reply, id 28158, seq 3,
    length 64

    But when I change the tunnel type to vxlan, ping fails.

    [root@vm1 ~]# ovs-vsctl del-port testbr0 gre0
    [root@vm1 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface
    vxlan0 type=vxlan options:local_ip=20.20.0.183
    options:remote_ip=30.30.0.193 options:dst_port=4789
    [root@vm1 ~]# ovs-vsctl show
    ff70c814-d1b0-4018-aee8-8b635187afee
        Bridge "testbr0"
            Port "testbr0"
    Interface "testbr0"
    type: internal
            Port "vxlan0"
    Interface "vxlan0"
    type: vxlan
    options: {dst_port="4789", local_ip="20.20.0.183",
    remote_ip="30.30.0.193"}
            Port "2cfb62a9b0f04_l"
    Interface "2cfb62a9b0f04_l"
    ovs_version: "2.9.2"

    [root@vm2 ~]# ovs-vsctl del-port testbr0 gre0
    [root@vm2 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface
    vxlan0 type=vxlan options:local_ip=30.30.0.193
    options:remote_ip=20.20.0.183 options:dst_port=4789
    [root@vm2 ~]# ovs-vsctl show
    b85514db-3f29-4f7a-9001-37d70adfca34
        Bridge "testbr0"
            Port "a0769422cfc04_l"
    Interface "a0769422cfc04_l"
            Port "vxlan0"
    Interface "vxlan0"
    type: vxlan
    options: {dst_port="4789", local_ip="30.30.0.193",
    remote_ip="20.20.0.183"}
            Port "testbr0"
    Interface "testbr0"
    type: internal
    ovs_version: "2.9.2"

    Ping fails with this setup

    [root@vm1 ~]# nsenter -n -t `docker inspect be4
    --format={{.State.Pid}}` -- ping 70.70.0.20
    PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data.
    ^C
    --- 70.70.0.20 ping statistics ---
    6 packets transmitted, 0 received, 100% packet loss, time 4999ms

    Expected traffic is not seen on VM2

    [root@vm2 ~]# tcpdump -n -i any host 20.20.0.183
    tcpdump: verbose output suppressed, use -v or -vv for full
    protocol decode
    listening on any, link-type LINUX_SLL (Linux cooked), capture
    size 262144 bytes
    ^C
    0 packets captured
    0 packets received by filter
    0 packets dropped by kernel

    Kindly let me know if you need more information.

    Siva Teja.

    On Tue, Nov 6, 2018 at 10:49 AM Gregory Rose
    <gvrose8...@gmail.com <mailto:gvrose8...@gmail.com>> wrote:


        On 11/5/2018 6:10 PM, Siva Teja ARETI wrote:
        Hi,

        I am trying to use local_ip option for a VXLAN tunnel using
        ovs but it does not seem to work. The same works when I use
        GRE tunnel. I also found a previous discussion from another
        user who tried the exact same approach. Here is the link to
        the discussion

        _https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html_

        I am unable to find any working resolution at the end of
        this discussion. Could you please help?

        I looked into that but was never able to set up a
        configuration like the one in that discussion and could
        not repro the bug.

        Please provide some details on your usage, configuration and
        steps to repro and I can look into it.

        Thanks,

        - Greg


        I am using ovs 2.9.2

        [root@localhost ~]# ovs-vsctl --version
        ovs-vsctl (Open vSwitch) 2.9.2
        DB Schema 7.15.1

        Thanks,
        Siva Teja.


        _______________________________________________
        discuss mailing list
        disc...@openvswitch.org <mailto:disc...@openvswitch.org>
        https://mail.openvswitch.org/mailman/listinfo/ovs-discuss



_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Reply via email to