[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198135#comment-15198135 ] Dan Osborne edited comment on MESOS-4823 at 3/16/16 8:55 PM: - Thank you for providing the example use case. Can you explain, on a technical level, on what conditions you are planning to trigger creation of these ip-tables rules? I'm concerned that the capability you're trying to provide makes a lot of assumptions about both the mesos cluster and the CNI network's configurations, and to what degree both are accessible by the public network. I believe that if this behavior goes in, to some degree it should be opt-in or opt-out, as not all clusters nor CNI network's would want such a behavior. Some counter use cases - 1. if the CNI network _is_ assigning publicly accessible addresses, the port mapping becomes a redundant. 2. if they are using a load balancer, they would not need port forwarding as the load balancer will forward public requests onto the private CNI network. was (Author: djosborne): Thank you for providing the example use case. Can you explain, on a technical level, what condition you are planning that will trigger creation of these ip-tables rules? I'm concerned that the capability you're trying to provide makes a lot of assumptions about both the mesos cluster and the CNI network's configurations, and to what degree both are accessible by the public network. I believe that if this behavior goes in, to some degree it should be opt-in or opt-out, as not all clusters nor CNI network's would want such a behavior. Some counter use cases - 1. if the CNI network _is_ assigning publicly accessible addresses, the port mapping becomes a redundant. 2. if they are using a load balancer, they would not need port forwarding as the load balancer will forward public requests onto the private CNI network. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish ports that micro-services are listening on, > to the outside world. When containers are running on bridged (or ptp) > networking this can be achieved by installing port forwarding rules on the > agent (using iptables). This can be done in the `network/cni` isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197927#comment-15197927 ] Avinash Sridharan edited comment on MESOS-4823 at 3/16/16 6:58 PM: --- [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea hear was to provide NAT capability, along with the ability for the containers to specify the ports (if requireD) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, and it seems like there are other folks that are interesting in tis requirement as well https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewalling rules to be part of the CNI spec. Currently however this is not the case and hence we will need to support it in the isolator. was (Author: avin...@mesosphere.io): [~djosborne] interesting point. I guess maybe the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea hear was to provide NAT capability, along with the ability for the containers to specify the ports (if requireD) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, and it seems like there are other folks that are interesting in tis requirement as well https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewalling rules to be part of the CNI spec. Currently however this is not the case and hence we will need to support it in the isolator. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish ports that micro-services are listening on, > to the outside world. When containers are running on bridged (or ptp) > networking this can be achieved by installing port forwarding rules on the > agent (using iptables). This can be done in the `network/cni` isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197927#comment-15197927 ] Avinash Sridharan edited comment on MESOS-4823 at 3/16/16 7:00 PM: --- [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea hear was to provide NAT capability, along with the ability for the containers to specify the ports (if requireD) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewall rules to be part of the CNI spec. Currently, however, this is not the case and hence we will need to support it in the isolator. was (Author: avin...@mesosphere.io): [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea hear was to provide NAT capability, along with the ability for the containers to specify the ports (if requireD) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, and it seems like there are other folks that are interesting in tis requirement as well https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewalling rules to be part of the CNI spec. Currently however this is not the case and hence we will need to support it in the isolator. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish ports that micro-services are listening on, > to the outside world. When containers are running on bridged (or ptp) > networking this can be achieved by installing port forwarding rules on the > agent (using iptables). This can be done in the `network/cni` isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197927#comment-15197927 ] Avinash Sridharan edited comment on MESOS-4823 at 3/16/16 10:32 PM: [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea here was to provide NAT capability, along with the ability for the containers to specify the ports (if required) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewall rules to be part of the CNI spec. Currently, however, this is not the case and hence we will need to support it in the isolator. was (Author: avin...@mesosphere.io): [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea hear was to provide NAT capability, along with the ability for the containers to specify the ports (if requireD) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewall rules to be part of the CNI spec. Currently, however, this is not the case and hence we will need to support it in the isolator. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish to expose ports that micro-services are > listening on, to the outside world. When containers are running on bridged > (or ptp) networking this can be achieved by installing port forwarding rules > on the agent (using iptables). This can be done in the `network/cni` > isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197927#comment-15197927 ] Avinash Sridharan edited comment on MESOS-4823 at 3/16/16 10:42 PM: [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea here was to provide NAT capability, along with the ability for the containers to specify the ports (if required) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanism to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewall rules to be part of the CNI spec. Currently, however, this is not the case and hence we will need to support it in the isolator. was (Author: avin...@mesosphere.io): [~djosborne] interesting point. I guess the ticket is a bit misleading. The fact that containers are addressable (layer 3 addressable) doesn't mean that their IP addresses are globally routeable. The idea here was to provide NAT capability, along with the ability for the containers to specify the ports (if required) on which they want to expose their service. While the CNI spec allows the IP masquerade option to be specified, it doesn't specify any mechanisms to specify port forwarding rules. This is particularly essential to support any EXPOSE primitives specified by the images (as with docker's EXPOSE primitives). I have raised this issue in the cni-dev mailing list as well, https://groups.google.com/forum/#!topic/cni-dev/FW3BCFJwAxY and it does seem like there are other folks interested in port forwarding and firewall rules to be part of the CNI spec. Currently, however, this is not the case and hence we will need to support it in the isolator. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish to expose ports that micro-services are > listening on, to the outside world. When containers are running on bridged > (or ptp) networking this can be achieved by installing port forwarding rules > on the agent (using iptables). This can be done in the `network/cni` > isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4823) Implement port forwarding in `network/cni` isolator
[ https://issues.apache.org/jira/browse/MESOS-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200365#comment-15200365 ] Avinash Sridharan edited comment on MESOS-4823 at 3/17/16 9:08 PM: --- [~lxpollitt] I completely agree with your observation that port mapping is just one of the ways to expose the service to the external world. If the service (layer 4 + layer 3) is completely addressable by the underlying CNI network and port mapping is not required that is perfectly fine. The only point I am trying to make is there are cases where port translation "might" be required by the container to make its service accessible, but unfortunately there is no way in CNI to communicate this information to the underlying plugin and hence we were thinking of implementing this piece in the isolator itself. It is an opt-in where the frameworks would specify whether they want port mapping or not. The idea here is that we should not be breaking the CNI spec, but at the same time we feel that the spec itself is evolving and we should try to compensate for the missing pieces. Would love to schedule a hangout if you would like to discuss further on this and get some closure as to whether this is an acceptable solution to enable port mapping in CNI or maybe come up with an alternate solution that does not touch the CNI isolator. was (Author: avin...@mesosphere.io): [~lxpollitt] I completely agree with your observation that port mapping is just one of the ways to expose the service to the external world. If the service (layer 4 + layer 3) is completely addressable by the underlying CNI network and port mapping is not required that is perfectly fine. The only point I am trying to make is there are cases where port translation "might" be required by the container to make its service accessible, but unfortunately there is no way in CNI to communicate this information to the underlying plugin and hence we were thinking of implementing this piece in the isolator itself. It is an opt-in where the frameworks would specify whether they want to port mapping or not. The idea here is that we should not be breaking the CNI spec, but at the same time we feel that the spec itself evolving and we should try to compensate for the missing pieces. Would love to schedule a hangout if you would like to discuss further on this and get some closure as to whether this is an acceptable solution to enable port mapping in CNI or maybe come up with an alternate solution that does not touch the CNI isolator. > Implement port forwarding in `network/cni` isolator > --- > > Key: MESOS-4823 > URL: https://issues.apache.org/jira/browse/MESOS-4823 > Project: Mesos > Issue Type: Task > Components: containerization > Environment: linux >Reporter: Avinash Sridharan >Assignee: Avinash Sridharan >Priority: Critical > Labels: mesosphere > > Most docker and appc images wish to expose ports that micro-services are > listening on, to the outside world. When containers are running on bridged > (or ptp) networking this can be achieved by installing port forwarding rules > on the agent (using iptables). This can be done in the `network/cni` > isolator. > The reason we would like this functionality to be implemented in the > `network/cni` isolator, and not a CNI plugin, is that the specifications > currently do not support specifying port forwarding rules. Further, to > install these rules the isolator needs two pieces of information, the exposed > ports and the IP address associated with the container. Bother are available > to the isolator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
