The consensus when I asked seemed to be that VIMAGE+jail was the right combination to give every container its own private loopback interface, so I tried to build that. I noticed a few things:
1) The kernel prints out a warning message at boot time that VIMAGE is "highly experimental". Should I be concerned about running this in production? 2) Stopping jails with virtual network stacks generates warnings from UMA about memory being leaked. 3) It wasn't clear (or documented anywhere that I could see) how to get the host network set up properly. Obviously I'm not going to have a vlan for every single jail, so it seemed like what most people were doing was "bridge" along with a bunch of "epair" interfaces. I ended up with the following: network_interfaces="lo0 bridge0 bce0" autobridge_interfaces="bridge0" autobridge_bridge0="bce0 epair0a epair1a" cloned_interfaces="bridge0 epair0 epair1" ifconfig_bridge0="inet [deleted] netmask 0xffffff00" ifconfig_bridge0_ipv6="inet6 [deleted] prefixlen 64 accept_rtadv" ifconfig_bce0="up" ifconfig_epair0a="up" ifconfig_epair1a="up" The net.link.bridge.inherit_mac sysctl, which is documented in bridge(4), doesn't appear to work; I haven't yet verified that I can create a /etc/start_if.bridge0 to set the MAC address manually without breaking something else. The IPv6 stack regularly prints "in6_if2idlen: unknown link type (209)" to the console, which is annoying, and IPv6 on the host doesn't entirely work -- it accepts router advertisements but then gives [ENETUNREACH] trying to actually send packets to the default gateway. (IPv6 to the jails *does* work!) In each of the jails I have to manually configure a MAC address using /etc/start_if.epairNb to ensure that it's globally unique, but then everything seems to work. Does this match up with what other people have been doing? Anything I've missed? Any patches I should pull up to make this setup more reliable before I roll it out in production? -GAWollman _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
