Re: [lxc-users] Fun with lxc.network.type=phys
On Mon, 19 Jan 2015, ScrumpyJack wrote: > I'd like to connect a physical interface from a host to a LXC container > guest like so: > > lxc.network.type=phys > > And then assign a routable IP/32 address to the LXC container for it to > "just work". > > The problem is that I don't have a spare "real" physical interface, so on > the host i create a "virtual" interface > > ip link add link eth0 mac0 type macvlan > > I now have a new virtual interface called mac0 with a separate mac address > in my host. I assign it a test IP and it can be pinged from outside the > host. > > I add the following details to the container's config file > > lxc.network.type=phys > lxc.network.flags = up > lxc.network.link = mac0 > lxc.network.name = eth1 > > > I boot my LXC guest, and as expected the mac0 virtual interface gets > passed on to the guest, as the guest has a new interface called eth1 with > exactly the same mac address as the randomly generated mac0 mac address > from the host, and the mac0 interface is no longer available in the host. > > But that's as far as it goes. Assigning the same test IP address to the > guest doesn't have the desired effect and the containers is unreachable. I > see the traffic coming into eth0 on the host, but that's it. The guest > doesn't seem to get the traffic with it's IP. > > I don't want to use bridging, veths or taps, or any method other than > physical. > hi again. I'm wondering if my setup is so silly that all as ignoring it :) Meanwhile, I'm trawling this mailing list and searching online and there is nothing i see that might be of any help. If anyone with knowledge of lxc networking would be kind enough to tell me whether I'm mad or not, then I could keep looking for give up on passing a macvlan host interface to the LXC physically. That would be most kind. ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
On Wed, Jan 21, 2015 at 3:31 PM, ScrumpyJack wrote: > On Mon, 19 Jan 2015, ScrumpyJack wrote: > > > I'd like to connect a physical interface from a host to a LXC container > > guest like so: > > > > lxc.network.type=phys > > > > And then assign a routable IP/32 address to the LXC container for it to > > "just work". > > > > The problem is that I don't have a spare "real" physical interface, so on > > the host i create a "virtual" interface > > > > ip link add link eth0 mac0 type macvlan > > hi again. I'm wondering if my setup is so silly that all as ignoring it :) > It is, to be frank. lxc already supports macvlan, so there's no need to create it manually and use phys. What is it that you're trying to achieve? If it's "just because I want to", then good luck. If it's "I want to to have /32 in the container", then there are other ways to do that. I deploy just that with veth and bridge. -- Fajar ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
On Wed, 21 Jan 2015, Fajar A. Nugraha wrote: > > It is, to be frank. lxc already supports macvlan, so there's no need to > create it manually and use phys. I have been reading more in macvlan support and it is now clearer. > If it's "I want to to have /32 in the container", then there are other ways > to do that. I deploy just that with veth and bridge. Yes, i want to give a /32 to a container. If i stick to lxc.network.type = macvlan lxc.network.flags = up lxc.network.link = eth0 lxc.network.name = eth1 lxc.network.ipv4 = 21.45.463.23/32 (fake IP, obvs) lxc.network.ipv4.gateway = 21.45.463.23 would you expect that to work? I'm trying not to have to do any NATing or any other configuration in my host for my container to get traffic destined for 21.45.463.23/32 ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
On Wed, Jan 21, 2015 at 7:09 PM, scrumpyjack wrote: > Yes, i want to give a /32 to a container. > > If i stick to > > lxc.network.type = macvlan > lxc.network.flags = up > lxc.network.link = eth0 > lxc.network.name = eth1 > lxc.network.ipv4 = 21.45.463.23/32 (fake IP, obvs) > lxc.network.ipv4.gateway = 21.45.463.23 > > would you expect that to work? > > Nope. Your main mistake is that you thought since the /32 IP works in the host (e.g. when used as "eth0:1"), it would automagically work inside the container, the host would simply "know" where to route the packet. It doesn't work that way. Network-wise, the host and the container are two separate entities, which might have a private link (i.e. thru a private bridge or something). The generic explanation of a working setup can be "stolen" from xen wiki: http://wiki.xen.org/wiki/Vif-route Basically they use a combination of /32, specific route, and proxy arp. I use a similar but slightly different method. This is on ubuntu server. The host has 100.0.0.10/24, router is on 100.0.0.1, the container is on 100.0.0.11 (fake IPs, of course). The host communicates with the container thru a PRIVATE bridge with IP 192.168.124.1 (note that this IP doesn't even have to be in the same network as host and container's IP) Relevant part of host's /etc/network/interfaces ### auto eth0 iface eth0 inet static address 100.0.0.10 netmask 255.255.255.0 gateway 100.0.0.1 # this part functions similar as proxy arp, force eth0 to accepts packets # destined for the container's IP using static arp up arp -i eth0 -Ds 100.0.0.11 eth0 pub || true # this is an internal bridge used to connect the host to the container auto br0 iface br0 inet manual bridge_ports none bridge_maxwait 0 bridge_stp off bridge_fd 0 # add specific route for the container IP up ip route add 100.0.0.11/32 dev br0 || true ### Relevant part of container config. Note that this only sets the bridge and persistent vif mac & name. ### lxc.network.type=veth lxc.network.link=br0 lxc.network.veth.pair=veth-c1-0 lxc.network.flags=up lxc.network.hwaddr = 00:16:3E:FD:46:25 ### Relevant part of container's /etc/network/interfaces ### auto eth0 iface eth0 inet static address 100.0.0.11 netmask 255.255.255.255 # force route for host's br0 up ip route add 192.168.124.1 dev eth0 # ... and use it for default route up ip route add default via 192.168.124.1 ### Relevant output of several commands in the host ### # ip route ... default via 100.0.0.1 dev eth0 100.0.0.0/24 dev eth0 proto kernel scope link src 100.0.0.10 100.0.0.11 dev br0 scope link ... # arp -n Address HWtype HWaddress Flags Mask Iface ... 100.0.0.11ether 00:16:3e:fd:46:25 C br0 100.0.0.11* MPeth0 ... # brctl show bridge name bridge id STP enabled interfaces ... br0 8000.feb01cb4ee91 no veth-c1-0 ... ### -- Fajar ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
On Wed, 21 Jan 2015, Fajar A. Nugraha wrote: > On Wed, Jan 21, 2015 at 7:09 PM, scrumpyjack wrote: > > > Yes, i want to give a /32 to a container. > This is on ubuntu server. The host has 100.0.0.10/24, router is on > 100.0.0.1, the container is on 100.0.0.11 (fake IPs, of course). > The host communicates with the container thru a PRIVATE bridge with IP > 192.168.124.1 (note that this IP doesn't even have to be in the same > network as host and container's IP) > > Relevant part of host's /etc/network/interfaces > ### > auto eth0 > iface eth0 inet static > address 100.0.0.10 > netmask 255.255.255.0 > gateway 100.0.0.1 > # this part functions similar as proxy arp, force eth0 to accepts packets > # destined for the container's IP using static arp > up arp -i eth0 -Ds 100.0.0.11 eth0 pub || true > > # this is an internal bridge used to connect the host to the container > auto br0 > iface br0 inet manual > bridge_ports none > bridge_maxwait 0 > bridge_stp off > bridge_fd 0 > # add specific route for the container IP > up ip route add 100.0.0.11/32 dev br0 || true > ### > > > Relevant part of container config. Note that this only sets the bridge and > persistent vif mac & name. > ### > lxc.network.type=veth > lxc.network.link=br0 > lxc.network.veth.pair=veth-c1-0 > lxc.network.flags=up > lxc.network.hwaddr = 00:16:3E:FD:46:25 > ### > > > Relevant part of container's /etc/network/interfaces > ### > auto eth0 > iface eth0 inet static > address 100.0.0.11 > netmask 255.255.255.255 > # force route for host's br0 > up ip route add 192.168.124.1 dev eth0 > # ... and use it for default route > up ip route add default via 192.168.124.1 Yup, thanks, this worked for me. I was trying to use macvlan and phys to avoid having to add my eth0 to the bridgeport and putting into promiscuois mode, which this solves thanks again! ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
scrumpyjack wrote, On 01/21/2015 01:09 PM: On Wed, 21 Jan 2015, Fajar A. Nugraha wrote: It is, to be frank. lxc already supports macvlan, so there's no need to create it manually and use phys. I have been reading more in macvlan support and it is now clearer. If it's "I want to to have /32 in the container", then there are other ways to do that. I deploy just that with veth and bridge. Yes, i want to give a /32 to a container. If i stick to lxc.network.type = macvlan lxc.network.flags = up lxc.network.link = eth0 lxc.network.name = eth1 lxc.network.ipv4 = 21.45.463.23/32 (fake IP, obvs) lxc.network.ipv4.gateway = 21.45.463.23 would you expect that to work? I'm trying not to have to do any NATing or any other configuration in my host for my container to get traffic destined for 21.45.463.23/32 This IP, even if it's a fake, is not a valid IP b/c 463 is > 255 ... ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
Re: [lxc-users] Fun with lxc.network.type=phys
On Wed, Jan 21, 2015 at 10:50 PM, ScrumpyJack wrote: > On Wed, 21 Jan 2015, Fajar A. Nugraha wrote: > >> On Wed, Jan 21, 2015 at 7:09 PM, scrumpyjack wrote: >> >> > Yes, i want to give a /32 to a container. >> Relevant part of container's /etc/network/interfaces >> ### >> auto eth0 >> iface eth0 inet static >> address 100.0.0.11 >> netmask 255.255.255.255 >> # force route for host's br0 >> up ip route add 192.168.124.1 dev eth0 >> # ... and use it for default route >> up ip route add default via 192.168.124.1 > > Yup, thanks, this worked for me. > I was trying to use macvlan and phys to avoid having to add my eth0 to the > bridgeport and putting into promiscuois mode, which this solves There's actually another alternative which does NOT involve bridging altogether, but requires an additional script for each container. It routes container traffic directly thru host side of container's veth interface, and use host's IP for container's default route. Note that you can actually use ANY IP that is in the host but not in the container. For example, since by default lxc creates a bridge called lxcbr0 with IP address 10.0.3.1, you can use that for container's default route, even when the container interface is not bridged to lxcbr0. Or you can use the host's public IP (which is what I used in this example). Something like this: Relevant part of host's /etc/network/interfaces ### auto eth0 iface eth0 inet static address 100.0.0.10 netmask 255.255.255.0 gateway 100.0.0.1 # this part functions similar as proxy arp, force eth0 to accepts packets # destined for the container's IP using static arp up arp -i eth0 -Ds 100.0.0.11 eth0 pub || true # note that you do NOT need any bridge in this setup ### Relevant part of container config. Note that this only sets persistent vif mac & name, but does NOT set any bridge. ### lxc.network.type=veth # in this setup you do NOT specify any bridge for lxc.network.link #lxc.network.link= lxc.network.veth.pair=veth-c1-0 lxc.network.flags=up lxc.network.hwaddr = 00:16:3E:FD:46:25 # this script will add specific route lxc.network.script.up = /path/to/your/container/network_up_script ### Content of /path/to/your/container/network_up_script (put it wherever you want, /var/lib/container_name/network_up.sh is a good place since each container will need its own script) ### #!/bin/bash # set specific route directly thru the host's side of container's veth interface /sbin/ip route add 100.0.0.11/32 dev veth-c1-0 ### Relevant part of container's /etc/network/interfaces ### auto eth0 iface eth0 inet static address 100.0.0.11 netmask 255.255.255.255 # force route for host's IP up ip route add 100.0.0.10 dev eth0 # ... and use it for default route up ip route add default via 100.0.0.10 ### -- Fajar ___ lxc-users mailing list lxc-users@lists.linuxcontainers.org http://lists.linuxcontainers.org/listinfo/lxc-users
