Things look fine. I'm not sure why the kernel would still be looking at
eth0 when you specifically told it to setup the tunnel on eth1...

I wonder if adding a static route for the multicast address to go to
eth1 would fix that somehow.


On Tue, Sep 26, 2017 at 12:48:01PM -0400, Ron Kelley wrote:
> Looks like I mistakenly said unicast.  In fact, we are using multicast group 
> IP (239.0.0.1) to setup our VXLAN interfaces (please see below).  Also, in 
> the example below, 10.250.1.21 is bound the eth0 (our management interface) 
> and 172.18.22.21 is bound to eth1 (VXLAN interface).  We have 8 servers 
> running thus far, and just increment the last octet to identify the server 
> (i.e.: .21=server-01, .22=server-02, etc).
> 
> Some more detailed information:
> 
> # --------------------------------------------------
> # The following script is run at boot time to setup VxLAN interfaces
> # /usr/local/bin/setup_vxlans.sh <up|down>
> # --------------------------------------------------
> #!/bin/bash
> 
> # Script to configure VxLAN networks
> ACTION="$1"
> 
> [ $ACTION == "up"   ]   && ip -4 route add 239.0.0.1 dev eth1
> [ $ACTION == "down"   ] && ip -4 route del 239.0.0.1 dev eth1
> 
> for i in {1101..1130}
> do
>    [ $ACTION == "up"   ]  && ip link add vxlan.${i} type vxlan group 
> 239.0.0.1 dev eth1 dstport 0 id ${i} && ip link set vxlan.${i} up
>    [ $ACTION == "down" ]  && ip link set vxlan.${i} down && ip link del 
> vxlan.${i}
> done
> # --------------------------------------------------
> 
> 
> 
> # --------------------------------------------------
> # The /etc/network/interfaces file:
> # --------------------------------------------------
> source /etc/network/interfaces.d/*
> 
> # The loopback network interface
> auto lo
> iface lo inet loopback
> 
> # The primary network interface
> auto eth0
> iface eth0 inet static
>       address 10.250.1.21
>       netmask 255.255.255.0
>       broadcast 10.251.1.255
>       gateway 10.251.1.1
>       dns-nameservers 8.8.8.8 4.2.2.2
> auto eth1
> iface eth1 inet static
>   address 172.17.22.21
>   netmask 255.255.255.0
> # --------------------------------------------------
> 
> 
> # --------------------------------------------------
> # Output from “lxc profile show <container>” snippet
> # --------------------------------------------------
> ...
>   limits.cpu: "2"
>   limits.memory: 2000MB
>   limits.memory.swap: “true"
> ...
> devices:
>   eth0:
>     name: eth0
>     nictype: macvlan
>     parent: vxlan.1105
>     type: nic
> ...
> # --------------------------------------------------
> 
> 
> Anything seem wrong or out of place?
> 
> Thanks,
> 
> -Ron
> 
> 
> 
> 
> > On Sep 26, 2017, at 10:59 AM, Stéphane Graber <stgra...@ubuntu.com> wrote:
> > 
> > Hi,
> > 
> > Ok, so you're doing your own VXLAN youtside of LXD and then connecting
> > containers directly to it.
> > 
> > The kernel error is very odd for unicast vxlan... there's really no
> > reason for it to ever use the other interface...
> > 
> > So I'm assuming the 10.250.1.21 IP is on eth0 and 172.18.22.21 on eth1
> > (or the reverse)? that is, you don't have both subnets on eth1.
> > 
> > On Tue, Sep 26, 2017 at 09:52:37AM -0400, Ron Kelley wrote:
> >> Thanks Stephane.
> >> 
> >> Here is a “lxc network list” on the hosts:
> >> 
> >> rkelley@LXD-QA-Server-04:~$ lxc network list
> >> +--------+----------+---------+-------------+---------+
> >> |  NAME  |   TYPE   | MANAGED | DESCRIPTION | USED BY |
> >> +--------+----------+---------+-------------+---------+
> >> | eth0   | physical | NO      |             | 0       |
> >> +--------+----------+---------+-------------+---------+
> >> | eth1   | physical | NO      |             | 2       |
> >> +--------+----------+---------+-------------+---------+
> >> | virbr0 | bridge   | NO      |             | 0       |
> >> +--------+----------+---------+-------------+————+
> >> 
> >> 
> >> Also, we are using vxlan in unicast mode via eth1.  Each LXD server has a 
> >> unicast IP address on eth1 that lives in a separate VLAN from eth0 on the 
> >> directly connected network switch.  If both eth0 and eth1 were in the same 
> >> VLAN, I could possible an issue.  Once a container is spun it, it is 
> >> attached to a VXLAN interface off eth1 (i.e.: vxlan.1115)
> >> 
> >> Thus, I am scratching my head..
> >> 
> >> -Ron
> >> 
> >> 
> >>> On Sep 26, 2017, at 9:02 AM, Stéphane Graber <stgra...@ubuntu.com> wrote:
> >>> 
> >>> On Sun, Sep 24, 2017 at 03:27:27PM -0400, Ron Kelley wrote:
> >>>> Greetings all,
> >>>> 
> >>>> Trying to isolate a condition whereby a container providing firewall 
> >>>> services seems to stop processing traffic for a short time.  We keep 
> >>>> getting the below information in /var/log/syslog of the server running 
> >>>> the firewall.  The IP addresses shown match the network interfaces of 
> >>>> the remote LXD server running the web server.  These IPs are for the 
> >>>> server itself, and not the container IP
> >>>> 
> >>>> Sep 24 15:10:25 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21  to 172.18.22.21
> >>>> Sep 24 15:10:26 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 
> >>>> Sep 24 15:10:27 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21  to 172.18.22.21
> >>>> Sep 24 15:10:28 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 
> >>>> Sep 24 15:10:29 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21  to 172.18.22.21
> >>>> Sep 24 15:10:30 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 
> >>>> Sep 24 15:10:31 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 10.250.1.21  to 172.18.22.21
> >>>> Sep 24 15:10:32 LXD-Server-04 kernel: [144272.412154] vxlan.1104: 
> >>>> 00:11:22:aa:66:a3 migrated from 172.18.22.21 to 10.250.1.21 
> >>>> 
> >>>> Notice how they migrate from one interface to another and then back 
> >>>> again.  Any idea as to why these messages are getting logged?
> >>>> 
> >>>> Thanks.
> >>>> 
> >>>> -Ron
> >>> 
> >>> Hmm, so I think I'm going to need a bit more details on the setup.
> >>> Can you show the "lxc network show" for the network on both hosts?
> >>> 
> >>> My current guess is that you're using vxlan in multicast mode and both
> >>> your hosts have two IPs on two subnets. Multicast VXLAN works on both
> >>> those subnets and it can therefore see the same remote MAC on both,
> >>> having it flip/flop between the two paths.
> >>> 
> >>> -- 
> >>> Stéphane Graber
> >>> Ubuntu developer
> >>> http://www.ubuntu.com
> >>> _______________________________________________
> >>> lxc-users mailing list
> >>> lxc-users@lists.linuxcontainers.org
> >>> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> 
> >> _______________________________________________
> >> lxc-users mailing list
> >> lxc-users@lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> > 
> > -- 
> > Stéphane Graber
> > Ubuntu developer
> > http://www.ubuntu.com
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users@lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
> 
> _______________________________________________
> lxc-users mailing list
> lxc-users@lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users

-- 
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com

Attachment: signature.asc
Description: PGP signature

_______________________________________________
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

Reply via email to