Re: sysctl behavior for docker-io
On Mon, Oct 7, 2013 at 3:47 PM, Richard W.M. Jones wrote: > Another way to look at it might be: Since a lot of people have libvirt > installed (it's the default isn't it?) and hence forwarding has been > on for many people for a long time, what harm is it causing? RFC 1812 > 2.2.8.1 Embedded Routers > > The embedded router feature seems to make building a network easy, but >it has a number of hidden pitfalls: > > (1) If a host has only a single constituent-network interface, it should not > act as a router. > > For example, hosts with embedded router code that gratuitously forward > broadcast packets or datagrams on the same net often cause packet avalanches. I'm almost certain that some other RFC requires forwarding for ~workstations to be opt-in, but I couldn't find it. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Sun, Oct 6, 2013 at 11:32 PM, Lennart Poettering
wrote:
> This is the general problem that IP forwarding is no local setting, and
> that the global setting has no inherent concept of ownership or
> refcounting.
The proper place for this seems to be firewalld, which should not only
control the individual sysctl, but also the more detailed forwarding
semantics (i.e. the application should request a specific, fairly
high-level forwarding scenario ("do a NAT of all traffic from
$this_ethernet and $this_wifi to $that ethernet"), and the firewall
should manage both iptables and sysctl.
I guess this is suggestion wouldn't be currently met with universal
approval, would it?
Mirek
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Mon, Oct 07, 2013 at 10:06:51AM +0100, Daniel P. Berrange wrote: > We really only wanted to enable forwarding from virbr0, to the LAN, but > you can't toggle this per NIC afaick - you have to turn on the global There seems to be per-NIC settings at: /proc/sys/net/ipv*/conf/*/forwarding Regards Till -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Mon, Oct 07, 2013 at 10:06:51AM +0100, Daniel P. Berrange wrote: > On Sun, Oct 06, 2013 at 07:25:50PM -0400, Matthew Miller wrote: > > On Sun, Oct 06, 2013 at 11:32:13PM +0200, Lennart Poettering wrote: > > > Or in other words: I don't think it makes much sense to turn this on > > > only at runtime inside the service file as matthew suggests, as it hides > > > the fact that the setting is made, makes it hard for admins to discover > > > and override it, and creates the assumption that the package would turn > > > off the setting safely again after the daemon exited, but which it > > > doesn't and can't since it doesn't know if anything else still requires > > > it. > > > Hope that makes some sense, > > > > It does make some sense; overall I don't think there's a really good answer > > here. In trying to figure out what's the most sensible given that, I looked > > at what libvirt does, which is turn it on globally in exactly the hidden way > > you suggest, and makes no attempt to restore it. I'm not really excited > > about that, but apparently that's been the case for a while. > > Yeah, what libvirt does is really not very nice. If you want to use a > routed networking setup though, I don't know of any better options for > making this work. > > We really only wanted to enable forwarding from virbr0, to the LAN, but > you can't toggle this per NIC afaick - you have to turn on the global > ip_forwarding sysctl. Libvirt just turns it on when first creating its > NAT'd device, which for most installs will be at boot time when libvirtd > starts. Another way to look at it might be: Since a lot of people have libvirt installed (it's the default isn't it?) and hence forwarding has been on for many people for a long time, what harm is it causing? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Sun, Oct 06, 2013 at 07:25:50PM -0400, Matthew Miller wrote: > On Sun, Oct 06, 2013 at 11:32:13PM +0200, Lennart Poettering wrote: > > Or in other words: I don't think it makes much sense to turn this on > > only at runtime inside the service file as matthew suggests, as it hides > > the fact that the setting is made, makes it hard for admins to discover > > and override it, and creates the assumption that the package would turn > > off the setting safely again after the daemon exited, but which it > > doesn't and can't since it doesn't know if anything else still requires > > it. > > Hope that makes some sense, > > It does make some sense; overall I don't think there's a really good answer > here. In trying to figure out what's the most sensible given that, I looked > at what libvirt does, which is turn it on globally in exactly the hidden way > you suggest, and makes no attempt to restore it. I'm not really excited > about that, but apparently that's been the case for a while. Yeah, what libvirt does is really not very nice. If you want to use a routed networking setup though, I don't know of any better options for making this work. We really only wanted to enable forwarding from virbr0, to the LAN, but you can't toggle this per NIC afaick - you have to turn on the global ip_forwarding sysctl. Libvirt just turns it on when first creating its NAT'd device, which for most installs will be at boot time when libvirtd starts. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Sun, Oct 06, 2013 at 11:32:13PM +0200, Lennart Poettering wrote: > Or in other words: I don't think it makes much sense to turn this on > only at runtime inside the service file as matthew suggests, as it hides > the fact that the setting is made, makes it hard for admins to discover > and override it, and creates the assumption that the package would turn > off the setting safely again after the daemon exited, but which it > doesn't and can't since it doesn't know if anything else still requires > it. > Hope that makes some sense, It does make some sense; overall I don't think there's a really good answer here. In trying to figure out what's the most sensible given that, I looked at what libvirt does, which is turn it on globally in exactly the hidden way you suggest, and makes no attempt to restore it. I'm not really excited about that, but apparently that's been the case for a while. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Fri, 04.10.13 16:04, Matthew Miller (mat...@fedoraproject.org) wrote: > On Fri, Oct 04, 2013 at 02:15:07PM -0500, Lokesh Mandvekar wrote: > > So, IP forwarding seems to be disabled by default in Fedora. docker-io > > requires IP forwarding enabled > > > > With respect to packaging, we'd like to have docker-io installation set > > sysctl values to enable IPv4 and IPv6 forwarding: > > https://bugzilla.redhat.com/show_bug.cgi?id=1011680 > > > > I was told on #fedora-devel that changing sysctl values during installation > > would spell trouble from a sysadmin's POV, so my plan was to install > > 80-docker.conf into /usr/lib/sysctl.d but not have the IP forwarding sysctl > > values take effect at install time. Would this be the right approach? > > I agree that they shouldn't be changed at RPM install time. However, I'm > also not sure that we should drop something into sysctl.d, because > > a) that doesn't take effect with the case of "yum install docker-io; >systemctl start docker", so that's confusing for users > > b) having docker _installed_ isn't really hte case where we need this -- >it's when docker is running. > > So, my first suggestion is to put the configuration into the systemd service > file. > > But, I have a question: What does libvirt do? Both as an example, and as a > possible solution -- will this problem go away when we convert to using > that, because libvirt will just take care of that? This is the general problem that IP forwarding is no local setting, and that the global setting has no inherent concept of ownership or refcounting. I am pretty sure it would suck if docker-io decides to take sole ownership of this setting, but I am not convinced that libvirt would be a much better owner. Given that this is a network related setting I have the suspicion that if something should take sole ownership of this setting then it probably should be some networking package, alas we have so many of those, and at least two are competing to be the one implementation that people use on servers to setup the network. Given the lack of an ownership concept, and thus no chance that the sysctl is dynamically reset to off as soon as no running service is requiring it anymore, I'd probably not bother at all with trying to do that, hence the best thing we could currently do is simply turn it on statically on install of each package that needs it (by dropping in individual tmpfiles in /usr/sysctl.d/ for each package that needs it), and documenting how the admin can reset it temporarily (with echo > to /proc/sys), or statically (by symlinking the tmpfiles snippets under identical names in /etc/sysctl.d/ to /dev/null). Or in other words: I don't think it makes much sense to turn this on only at runtime inside the service file as matthew suggests, as it hides the fact that the setting is made, makes it hard for admins to discover and override it, and creates the assumption that the package would turn off the setting safely again after the daemon exited, but which it doesn't and can't since it doesn't know if anything else still requires it. Hope that makes some sense, Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Fri, Oct 04, 2013 at 03:21:07PM -0500, Lokesh Mandvekar wrote: > On Fri, Oct 04, 2013 at 04:04:19PM -0400, Matthew Miller wrote: > > On Fri, Oct 04, 2013 at 02:15:07PM -0500, Lokesh Mandvekar wrote: > > > So, IP forwarding seems to be disabled by default in Fedora. docker-io > > > requires IP forwarding enabled > > > > > > With respect to packaging, we'd like to have docker-io installation set > > > sysctl values to enable IPv4 and IPv6 forwarding: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1011680 > > > > > > I was told on #fedora-devel that changing sysctl values during > > > installation > > > would spell trouble from a sysadmin's POV, so my plan was to install > > > 80-docker.conf into /usr/lib/sysctl.d but not have the IP forwarding > > > sysctl > > > values take effect at install time. Would this be the right approach? > > > > I agree that they shouldn't be changed at RPM install time. However, I'm > > also not sure that we should drop something into sysctl.d, because > > > > a) that doesn't take effect with the case of "yum install docker-io; > >systemctl start docker", so that's confusing for users > > > > b) having docker _installed_ isn't really hte case where we need this -- > >it's when docker is running. > > > > So, my first suggestion is to put the configuration into the systemd service > > file. > > > > But, I have a question: What does libvirt do? Both as an example, and as a > > possible solution -- will this problem go away when we convert to using > > that, because libvirt will just take care of that? > > Josh (cc'd) said libvirtd would enable it, but we still need to take care of > this for docker+lxc. I agree with Matthew that the unit file is a good place to do it. Another option would be to enable it from the docker daemon itself. That way all the other distros wouldn't have to hit this same issue when packaging docker. Josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Fri, Oct 04, 2013 at 03:21:07PM -0500, Lokesh Mandvekar wrote: > Josh (cc'd) said libvirtd would enable it, but we still need to take care of > this for docker+lxc. If we pick libvirt-lxc as the preferred configuration, we can maybe get away with just documenting changes needed if you want to use the other tools. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Fri, Oct 04, 2013 at 04:04:19PM -0400, Matthew Miller wrote: > On Fri, Oct 04, 2013 at 02:15:07PM -0500, Lokesh Mandvekar wrote: > > So, IP forwarding seems to be disabled by default in Fedora. docker-io > > requires IP forwarding enabled > > > > With respect to packaging, we'd like to have docker-io installation set > > sysctl values to enable IPv4 and IPv6 forwarding: > > https://bugzilla.redhat.com/show_bug.cgi?id=1011680 > > > > I was told on #fedora-devel that changing sysctl values during installation > > would spell trouble from a sysadmin's POV, so my plan was to install > > 80-docker.conf into /usr/lib/sysctl.d but not have the IP forwarding sysctl > > values take effect at install time. Would this be the right approach? > > I agree that they shouldn't be changed at RPM install time. However, I'm > also not sure that we should drop something into sysctl.d, because > > a) that doesn't take effect with the case of "yum install docker-io; >systemctl start docker", so that's confusing for users > > b) having docker _installed_ isn't really hte case where we need this -- >it's when docker is running. > > So, my first suggestion is to put the configuration into the systemd service > file. > > But, I have a question: What does libvirt do? Both as an example, and as a > possible solution -- will this problem go away when we convert to using > that, because libvirt will just take care of that? Josh (cc'd) said libvirtd would enable it, but we still need to take care of this for docker+lxc. > > -- > Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ > -- > devel mailing list > devel@lists.fedoraproject.org > https://admin.fedoraproject.org/mailman/listinfo/devel > Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct -- Lokesh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: sysctl behavior for docker-io
On Fri, Oct 04, 2013 at 02:15:07PM -0500, Lokesh Mandvekar wrote: > So, IP forwarding seems to be disabled by default in Fedora. docker-io > requires IP forwarding enabled > > With respect to packaging, we'd like to have docker-io installation set > sysctl values to enable IPv4 and IPv6 forwarding: > https://bugzilla.redhat.com/show_bug.cgi?id=1011680 > > I was told on #fedora-devel that changing sysctl values during installation > would spell trouble from a sysadmin's POV, so my plan was to install > 80-docker.conf into /usr/lib/sysctl.d but not have the IP forwarding sysctl > values take effect at install time. Would this be the right approach? I agree that they shouldn't be changed at RPM install time. However, I'm also not sure that we should drop something into sysctl.d, because a) that doesn't take effect with the case of "yum install docker-io; systemctl start docker", so that's confusing for users b) having docker _installed_ isn't really hte case where we need this -- it's when docker is running. So, my first suggestion is to put the configuration into the systemd service file. But, I have a question: What does libvirt do? Both as an example, and as a possible solution -- will this problem go away when we convert to using that, because libvirt will just take care of that? -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
