Things look fine. I'm not sure why the kernel would still be looking at eth0 when you specifically told it to setup the tunnel on eth1...
I wonder if adding a static route for the multicast address to go to eth1 would fix that somehow. On Tue, Sep 26, 2017 at 12:48:01PM -0400, Ron Kelley wrote: > Looks like I mistakenly said unicast. In fact, we are using multicast group > IP (239.0.0.1) to setup our VXLAN interfaces (please see below). Also, in > the example below, 10.250.1.21 is bound the eth0 (our management interface) > and 172.18.22.21 is bound to eth1 (VXLAN interface). We have 8 servers > running thus far, and just increment the last octet to identify the server > (i.e.: .21=server-01, .22=server-02, etc). > > Some more detailed information: > > # -------------------------------------------------- > # The following script is run at boot time to setup VxLAN interfaces > # /usr/local/bin/setup_vxlans.sh <up|down> > # -------------------------------------------------- > #!/bin/bash > > # Script to configure VxLAN networks > ACTION="$1" > > [ $ACTION == "up" ] && ip -4 route add 239.0.0.1 dev eth1 > [ $ACTION == "down" ] && ip -4 route del 239.0.0.1 dev eth1 > > for i in {1101..1130} > do > [ $ACTION == "up" ] && ip link add vxlan.${i} type vxlan group > 239.0.0.1 dev eth1 dstport 0 id ${i} && ip link set vxlan.${i} up > [ $ACTION == "down" ] && ip link set vxlan.${i} down && ip link del > vxlan.${i} > done > # -------------------------------------------------- > > > > # -------------------------------------------------- > # The /etc/network/interfaces file: > # -------------------------------------------------- > source /etc/network/interfaces.d/* > > # The loopback network interface > auto lo > iface lo inet loopback > > # The primary network interface > auto eth0 > iface eth0 inet static > address 10.250.1.21 > netmask 255.255.255.0 > broadcast 10.251.1.255 > gateway 10.251.1.1 > dns-nameservers 8.8.8.8 4.2.2.2 > auto eth1 > iface eth1 inet static > address 172.17.22.21 > netmask 255.255.255.0 > # -------------------------------------------------- > > > # -------------------------------------------------- > # Output from “lxc profile show <container>” snippet > # -------------------------------------------------- > ... > limits.cpu: "2" > limits.memory: 2000MB > limits.memory.swap: “true" > ... > devices: > eth0: > name: eth0 > nictype: macvlan > parent: vxlan.1105 > type: nic > ... > # -------------------------------------------------- > > > Anything seem wrong or out of place? > > Thanks, > > -Ron > > > > > > On Sep 26, 2017, at 10:59 AM, Stéphane Graber <stgra...@ubuntu.com> wrote: > > > > Hi, > > > > Ok, so you're doing your own VXLAN youtside of LXD and then connecting > > containers directly to it. > > > > The kernel error is very odd for unicast vxlan... there's really no > > reason for it to ever use the other interface... > > > > So I'm assuming the 10.250.1.21 IP is on eth0 and 172.18.22.21 on eth1 > > (or the reverse)? that is, you don't have both subnets on eth1. > > > > On Tue, Sep 26, 2017 at 09:52:37AM -0400, Ron Kelley wrote: > >> Thanks Stephane. > >> > >> Here is a “lxc network list” on the hosts: > >> > >> rkelley@LXD-QA-Server-04:~$ lxc network list > >> +--------+----------+---------+-------------+---------+ > >> | NAME | TYPE | MANAGED | DESCRIPTION | USED BY | > >> +--------+----------+---------+-------------+---------+ > >> | eth0 | physical | NO | | 0 | > >> +--------+----------+---------+-------------+---------+ > >> | eth1 | physical | NO | | 2 | > >> +--------+----------+---------+-------------+---------+ > >> | virbr0 | bridge | NO | | 0 | > >> +--------+----------+---------+-------------+————+ > >> > >> > >> Also, we are using vxlan in unicast mode via eth1. Each LXD server has a > >> unicast IP address on eth1 that lives in a separate VLAN from eth0 on the > >> directly connected network switch. If both eth0 and eth1 were in the same > >> VLAN, I could possible an issue. Once a container is spun it, it is > >> attached to a VXLAN interface off eth1 (i.e.: vxlan.1115) > >> > >> Thus, I am scratching my head.. > >> > >> -Ron > >> > >> > >>> On Sep 26, 2017, at 9:02 AM, Stéphane Graber <stgra...@ubuntu.com> wrote: > >>> > >>> On Sun, Sep 24, 2017 at 03:27:27PM -0400, Ron Kelley wrote: > >>>> Greetings all, > >>>> > >>>> Trying to isolate a condition whereby a container providing firewall > >>>> services seems to stop processing traffic for a short time. We keep > >>>> getting the below information in /var/log/syslog of the server running > >>>> the firewall. The IP addresses shown match the network interfaces of > >>>> the remote LXD server running the web server. These IPs are for the > >>>> server itself, and not the container IP > >>>> > >>>> Sep 24 15:10:25 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21 to 172.18.22.21 > >>>> Sep 24 15:10:26 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 > >>>> Sep 24 15:10:27 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21 to 172.18.22.21 > >>>> Sep 24 15:10:28 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 > >>>> Sep 24 15:10:29 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21 to 172.18.22.21 > >>>> Sep 24 15:10:30 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 > >>>> Sep 24 15:10:31 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21 to 172.18.22.21 > >>>> Sep 24 15:10:32 LXD-Server-04 kernel: [144272.412154] vxlan.1104: > >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 > >>>> > >>>> Notice how they migrate from one interface to another and then back > >>>> again. Any idea as to why these messages are getting logged? > >>>> > >>>> Thanks. > >>>> > >>>> -Ron > >>> > >>> Hmm, so I think I'm going to need a bit more details on the setup. > >>> Can you show the "lxc network show" for the network on both hosts? > >>> > >>> My current guess is that you're using vxlan in multicast mode and both > >>> your hosts have two IPs on two subnets. Multicast VXLAN works on both > >>> those subnets and it can therefore see the same remote MAC on both, > >>> having it flip/flop between the two paths. > >>> > >>> -- > >>> Stéphane Graber > >>> Ubuntu developer > >>> http://www.ubuntu.com > >>> _______________________________________________ > >>> lxc-users mailing list > >>> lxc-users@lists.linuxcontainers.org > >>> http://lists.linuxcontainers.org/listinfo/lxc-users > >> > >> _______________________________________________ > >> lxc-users mailing list > >> lxc-users@lists.linuxcontainers.org > >> http://lists.linuxcontainers.org/listinfo/lxc-users > > > > -- > > Stéphane Graber > > Ubuntu developer > > http://www.ubuntu.com > > _______________________________________________ > > lxc-users mailing list > > lxc-users@lists.linuxcontainers.org > > http://lists.linuxcontainers.org/listinfo/lxc-users > > _______________________________________________ > lxc-users mailing list > lxc-users@lists.linuxcontainers.org > http://lists.linuxcontainers.org/listinfo/lxc-users -- Stéphane Graber Ubuntu developer http://www.ubuntu.com
signature.asc
Description: PGP signature
_______________________________________________ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users