Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

Hello,

sorry to insist, is the understanding below correct ? I'm really not sure.

I understand that network/portmapping isolator is using disjoint port 
ranges to multiplex traffic into the same ports into containers but I'm 
not really sure if we're talking about ephemeral or non-ephemeral ports 
here neither if I correctly understand what kind of port is for what 
kind of use.


About the direct mapping : what in the container is listening to the 
mapped port ? How ?


Also what kind of this ephemeral vs non-ephermeral port is called a 
hostPort in Marathon ?


Here's my initial understanding :

Thanks.
On 04/05/2017 12:23 PM, Thomas HUMMEL wrote:

Ok, thanks.

So if I wrap my head around all of this and try to answer my original 
question I come up with the following understanding :


- servicePorts a a Marathon only concept

- port mapping isolator is not compatible with docker containerizer

- port mapping isolator is useful when you cannot afford one ip / 
container


- port mapping isolator uses *ephemeral* ports to multiplex traffic 
into containers


the *ephemeral* port range is divided into *disjoint*  subsets of 
*contiguous* ports, each one affected to one container with a direct 
mapping hostport <-> containerPort.


- non-ephemeral ports are affected to framework as a resource. So 
containers have *disjoint* sets of them but *not in a contiguous* range


- the default port range offered by a slave is [31000-32000] : those 
are *non-ephemeral* ports and is not related to the activation or non 
activation of the port-mapping isolator


- with docker containerizer in HOST mode, Marathon framework is 
offered such a port (in the [31000-32000] range and shows it in the 
GUI, but the app can bind to any different hostport *not in that 
range* (ex: 9090). In BRIDGE mode, the Marathon so-called 'hostPort' 
has to be in that range (why is that ?)



I am right this time ? ;-)


Thanks

--

TH

Re: Mesos (and Marathon) port mapping

[Avinash Sridharan]

On Fri, Apr 7, 2017 at 6:36 AM, Thomas HUMMEL 
wrote:

> On 03/31/2017 07:51 PM, Jie Yu wrote:
>
>> Tomek and Olivier,
>>
>> The bridge network support (with port mapping) has been added to Mesos
>> 1.2. See this doc for more details how to use it:
>> https://github.com/apache/mesos/blob/master/docs/cni.md#a-
>> port-mapper-plugin-for-cni-networks
>>
>> TL;DR: we developed a CNI port mapper plugin (DNAT) in Mesos repo, and
>> uses a delegation model in CNI. For the bridge CNI plugin, you can simply
>> use the default bridge plugin in CNI repo (https://github.com/containern
>> etworking/cni). @avinash can explain more here.
>>
>
> Am I right thinking we can achieve the same as
> MesosContainerizer/CNI/Bridge network plugin wrapped by the port mapper
> plugin with the DockerContainerizer ?
>
> Yes, MesosContainerizer/CNI/Bridge network plugin + CNI port-mapper plugin
is identical to the docker's BRIDGE mode networking.

> Thanks.
>
> --
> TH
>



-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

On 03/31/2017 07:51 PM, Jie Yu wrote:

Tomek and Olivier,

The bridge network support (with port mapping) has been added to Mesos 
1.2. See this doc for more details how to use it:

https://github.com/apache/mesos/blob/master/docs/cni.md#a-port-mapper-plugin-for-cni-networks

TL;DR: we developed a CNI port mapper plugin (DNAT) in Mesos repo, and 
uses a delegation model in CNI. For the bridge CNI plugin, you can 
simply use the default bridge plugin in CNI repo 
(https://github.com/containernetworking/cni). @avinash can explain 
more here.


Am I right thinking we can achieve the same as 
MesosContainerizer/CNI/Bridge network plugin wrapped by the port mapper 
plugin with the DockerContainerizer ?


Thanks.

--
TH

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

Ok, thanks.

So if I wrap my head around all of this and try to answer my original 
question I come up with the following understanding :


- servicePorts a a Marathon only concept

- port mapping isolator is not compatible with docker containerizer

- port mapping isolator is useful when you cannot afford one ip / container

- port mapping isolator uses *ephemeral* ports to multiplex traffic into 
containers


the *ephemeral* port range is divided into *disjoint*  subsets of 
*contiguous* ports, each one affected to one container with a direct 
mapping hostport <-> containerPort.


- non-ephemeral ports are affected to framework as a resource. So 
containers have *disjoint* sets of them but *not in a contiguous* range


- the default port range offered by a slave is [31000-32000] : those are 
*non-ephemeral* ports and is not related to the activation or non 
activation of the port-mapping isolator


- with docker containerizer in HOST mode, Marathon framework is offered 
such a port (in the [31000-32000] range and shows it in the GUI, but the 
app can bind to any different hostport *not in that range* (ex: 9090). 
In BRIDGE mode, the Marathon so-called 'hostPort' has to be in that 
range (why is that ?)



I am right this time ? ;-)


Thanks

--

TH

Re: Mesos (and Marathon) port mapping

[Jie Yu]

Thomas,

- it is the hostports which are used to multiplex traffic into container.
> My understanding is that, since each container is in it's network
> namespace, it has its own full range of container ports and that you use a
> direct mapping (hostport n <-> same container port n), is that correct ?

Yes.

- those ports which are divided into disjoint subsets are the ephermeral
> ports. The non- ephemeral ports are in a set shared between all containers,
> correct ?


No. non-ephemeral ports are allocated by framework. (non-ephemeral ports
are modeled as Resources in Mesos). So containers must have disjoint sets
of non-ephemeral ports.

- the use case you described is when you cannot afford one ip/container and
> when you are using the mesos containeraizer : does it mean that network
> mapping isolation makes no sense with the docker containerizer or can it be
> somehow composed with it ?]


If you're looking for private bridge + DNAT solution (like Docker
--net=bridge), you can follow the following docs if you want to use it with
Mesos containerizer. It's supported through a more standard interface
called CNI (https://github.com/containernetworking/cni)
https://github.com/apache/mesos/blob/master/docs/cni.md
https://github.com/apache/mesos/blob/master/docs/cni.md#a-port-mapper-plugin-for-cni-networks

The ip/container limitation is not related to which containerizer you're
using. It's specific to the company (Twitter)'s environment. For instance,
we cannot change the service discovery mechanism at that time, requiring
all container's IP must be routable.

I didn't quite understand why you cannot use NAT (in the same way docker in
> BRIDGE mode does) and assign as many ip addresses that you want in a
> private network...


See my response above. If you're looking for docker --net=bridge support,
follow the two links above.

- Jie

On Fri, Mar 31, 2017 at 3:39 AM, Thomas HUMMEL 
wrote:

> Thanks for your answer,
>
> I've watched your talk. Very interesting.
>
> Let me check if I get everything staight :
>
> - it is the hostports which are used to multiplex traffic into container.
> My understanding is that, since each container is in it's network
> namespace, it has its own full range of container ports and that you use a
> direct mapping (hostport n <-> same container port n), is that correct ?
>
> - those ports which are divided into disjoint subsets are the ephermeral
> ports. The non- ephemeral ports are in a set shared between all containers,
> correct ?
>
> - the use case you described is when you cannot afford one ip/container
> and when you are using the mesos containeraizer : does it mean that network
> mapping isolation makes no sense with the docker containerizer or can it be
> somehow composed with it ?]
>
> I didn't quite understand why you cannot use NAT (in the same way docker
> in BRIDGE mode does) and assign as many ip addresses that you want in a
> private network...
>
> Thanks.
>
> --
>
> TH.
>
>
>
>

Re: Mesos (and Marathon) port mapping

[Jie Yu]

Tomek and Olivier,

The bridge network support (with port mapping) has been added to Mesos 1.2.
See this doc for more details how to use it:
https://github.com/apache/mesos/blob/master/docs/cni.md#a-port-mapper-plugin-for-cni-networks

TL;DR: we developed a CNI port mapper plugin (DNAT) in Mesos repo, and uses
a delegation model in CNI. For the bridge CNI plugin, you can simply use
the default bridge plugin in CNI repo (
https://github.com/containernetworking/cni). @avinash can explain more here.



On Fri, Mar 31, 2017 at 3:40 AM, Olivier Sallou 
wrote:

>
>
> On 03/31/2017 10:23 AM, Tomek Janiszewski wrote:
>
> I have a question that is related to this topic. In "docker support and
> current limitations" section [1] there is a following statement:
> > Only host network is supported. We will add bridge network support soon
> using CNI support in Mesos (MESOS-4641
> )
> Mentioned issue is resolved. Does this means bridge network is working for
> Mesos containerizer?
>
> [1]: https://github.com/apache/mesos/blob/master/docs/
> container-image.md#docker-support-and-current-limitations
>
> CNI support in unified containerizer (mesos) gives the possibility to
> assign an IP per container, so no port mapping (the ports you use will be
> used direclty as container has its own IP address). There is no "bridge"
> network as per Docker (mapping of container port 80 to host port 3 for
> example)
>
> Olivier
>
>
> pt., 31 mar 2017 o 02:04 użytkownik Jie Yu  napisał:
>
>> are you talking about the NAT feature of docker in BRIDGE m
>>
>>
>> Yes
>>
>>  - regarding the "port mapping isolator giving network namespace" : what
>> confuses me is that, given the previous answers, I thought that in that
>> case, the non-ephemeral port range was *shared* (as a ressource) between
>> containers, which sounds to me at the opposite of the namespace concept (as
>> a slightly different example 2 docker container have their own private 80
>> port for instance).
>>
>>
>> The port mapping isolator is for the case where ip per container is not
>> possible (due to ipam restriction, etc), but the user still wants to have
>> network namespace per container (for isolation, getting statistics, etc.)
>>
>> Since all containers, even if they are in separate namespaces, share the
>> same IP, we have to use some other mechanism to tell which packet belongs
>> to which container. We use ports in that case. You can find more details
>> about port mapping isolator in this talk I gave in 2015 MesosCon:
>> https://www.youtube.com/watch?v=ZA96g1M4v8Y
>>
>> - Jie
>>
>> On Thu, Mar 30, 2017 at 2:13 AM, Thomas HUMMEL 
>> wrote:
>>
>>
>> On 03/29/2017 07:25 PM, Jie Yu wrote:
>>
>> Thomas,
>>
>> I think you are confused about the port mapping for NAT purpose, and the port
>> mapping isolator
>> .
>> Those two very different thing. The port mapping isolator (unfortunate
>> naming), as described in the doc, gives you network namespace per container
>> without requiring ip per container. No NAT is involved. I think for you
>> case, you should not use it and it does not work for DockerContainerizer.
>>
>> Thanks,
>>
>> I'm not sure to understand what you say :
>>
>> - are you talking about the NAT feature of docker in BRIDGE mode ?
>>
>> - regarding the "port mapping isolator giving network namespace" : what
>> confuses me is that, given the previous answers, I thought that in that
>> case, the non-ephemeral port range was *shared* (as a ressource) between
>> containers, which sounds to me at the opposite of the namespace concept (as
>> a slightly different example 2 docker container have their own private 80
>> port for instance).
>>
>> What am I missing ?
>>
>> Thanks
>>
>> --
>> TH
>>
>>
>>
> --
> Olivier Sallou
> IRISA / University of Rennes 1
> Campus de Beaulieu, 35000 RENNES - FRANCE
> Tel: 02.99.84.71.95
>
> gpg key id: 4096R/326D8438  (keyring.debian.org)
> Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438
>
>
>

Re: Mesos (and Marathon) port mapping

[Olivier Sallou]

On 03/31/2017 10:23 AM, Tomek Janiszewski wrote:
> I have a question that is related to this topic. In "docker support
> and current limitations" section [1] there is a following statement:
> > Only host network is supported. We will add bridge network support
> soon using CNI support in Mesos (MESOS-4641
> )
> Mentioned issue is resolved. Does this means bridge network is working
> for Mesos containerizer?
>
> [1]: 
> https://github.com/apache/mesos/blob/master/docs/container-image.md#docker-support-and-current-limitations
CNI support in unified containerizer (mesos) gives the possibility to
assign an IP per container, so no port mapping (the ports you use will
be used direclty as container has its own IP address). There is no
"bridge" network as per Docker (mapping of container port 80 to host
port 3 for example)

Olivier
>
> pt., 31 mar 2017 o 02:04 użytkownik Jie Yu  > napisał:
>
> are you talking about the NAT feature of docker in BRIDGE m
>
>
> Yes
>
>  - regarding the "port mapping isolator giving network
> namespace" : what confuses me is that, given the previous
> answers, I thought that in that case, the
> non-ephemeral port range was *shared* (as a ressource) between
> containers, which sounds to me at the opposite of the
> namespace concept (as a slightly different example 2 docker
> container have their own private 80 port for instance).
>
>
> The port mapping isolator is for the case where ip per container
> is not possible (due to ipam restriction, etc), but the user still
> wants to have network namespace per container (for isolation,
> getting statistics, etc.)
>
> Since all containers, even if they are in separate namespaces,
> share the same IP, we have to use some other mechanism to tell
> which packet belongs to which container. We use ports in that
> case. You can find more details about port mapping isolator in
> this talk I gave in 2015
> MesosCon: https://www.youtube.com/watch?v=ZA96g1M4v8Y
>
> - Jie
>
> On Thu, Mar 30, 2017 at 2:13 AM, Thomas HUMMEL
> mailto:thomas.hum...@pasteur.fr>> wrote:
>
>
> On 03/29/2017 07:25 PM, Jie Yu wrote:
>> Thomas,
>>
>> I think you are confused about the port mapping for NAT
>> purpose, and the port mapping isolator
>> 
>> .
>> Those two very different thing. The port mapping isolator
>> (unfortunate naming), as described in the doc, gives you
>> network namespace per container without requiring ip per
>> container. No NAT is involved. I think for you case, you
>> should not use it and it does not work for DockerContainerizer.
> Thanks,
>
> I'm not sure to understand what you say :
>
> - are you talking about the NAT feature of docker in BRIDGE mode ?
>
> - regarding the "port mapping isolator giving network
> namespace" : what confuses me is that, given the previous
> answers, I thought that in that case, the non-ephemeral port
> range was *shared* (as a ressource) between containers, which
> sounds to me at the opposite of the namespace concept (as a
> slightly different example 2 docker container have their own
> private 80 port for instance).
>
> What am I missing ?
>
> Thanks
>
> --
> TH
>
>

-- 
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

Thanks for your answer,

I've watched your talk. Very interesting.

Let me check if I get everything staight :

- it is the hostports which are used to multiplex traffic into 
container. My understanding is that, since each container is in it's 
network namespace, it has its own full range of container ports and that 
you use a direct mapping (hostport n <-> same container port n), is that 
correct ?


- those ports which are divided into disjoint subsets are the ephermeral 
ports. The non- ephemeral ports are in a set shared between all 
containers, correct ?


- the use case you described is when you cannot afford one ip/container 
and when you are using the mesos containeraizer : does it mean that 
network mapping isolation makes no sense with the docker containerizer 
or can it be somehow composed with it ?]


I didn't quite understand why you cannot use NAT (in the same way docker 
in BRIDGE mode does) and assign as many ip addresses that you want in a 
private network...


Thanks.

--

TH.

Re: Mesos (and Marathon) port mapping

[Tomek Janiszewski]

I have a question that is related to this topic. In "docker support and
current limitations" section [1] there is a following statement:
> Only host network is supported. We will add bridge network support soon
using CNI support in Mesos (MESOS-4641
)
Mentioned issue is resolved. Does this means bridge network is working for
Mesos containerizer?

[1]:
https://github.com/apache/mesos/blob/master/docs/container-image.md#docker-support-and-current-limitations

pt., 31 mar 2017 o 02:04 użytkownik Jie Yu  napisał:

> are you talking about the NAT feature of docker in BRIDGE m
>
>
> Yes
>
>  - regarding the "port mapping isolator giving network namespace" : what
> confuses me is that, given the previous answers, I thought that in that
> case, the non-ephemeral port range was *shared* (as a ressource) between
> containers, which sounds to me at the opposite of the namespace concept (as
> a slightly different example 2 docker container have their own private 80
> port for instance).
>
>
> The port mapping isolator is for the case where ip per container is not
> possible (due to ipam restriction, etc), but the user still wants to have
> network namespace per container (for isolation, getting statistics, etc.)
>
> Since all containers, even if they are in separate namespaces, share the
> same IP, we have to use some other mechanism to tell which packet belongs
> to which container. We use ports in that case. You can find more details
> about port mapping isolator in this talk I gave in 2015 MesosCon:
> https://www.youtube.com/watch?v=ZA96g1M4v8Y
>
> - Jie
>
> On Thu, Mar 30, 2017 at 2:13 AM, Thomas HUMMEL 
> wrote:
>
>
> On 03/29/2017 07:25 PM, Jie Yu wrote:
>
> Thomas,
>
> I think you are confused about the port mapping for NAT purpose, and the port
> mapping isolator
> .
> Those two very different thing. The port mapping isolator (unfortunate
> naming), as described in the doc, gives you network namespace per container
> without requiring ip per container. No NAT is involved. I think for you
> case, you should not use it and it does not work for DockerContainerizer.
>
> Thanks,
>
> I'm not sure to understand what you say :
>
> - are you talking about the NAT feature of docker in BRIDGE mode ?
>
> - regarding the "port mapping isolator giving network namespace" : what
> confuses me is that, given the previous answers, I thought that in that
> case, the non-ephemeral port range was *shared* (as a ressource) between
> containers, which sounds to me at the opposite of the namespace concept (as
> a slightly different example 2 docker container have their own private 80
> port for instance).
>
> What am I missing ?
>
> Thanks
>
> --
> TH
>
>
>

Re: Mesos (and Marathon) port mapping

[Jie Yu]

>
> are you talking about the NAT feature of docker in BRIDGE m


Yes

 - regarding the "port mapping isolator giving network namespace" : what
> confuses me is that, given the previous answers, I thought that in that
> case, the non-ephemeral port range was *shared* (as a ressource) between
> containers, which sounds to me at the opposite of the namespace concept (as
> a slightly different example 2 docker container have their own private 80
> port for instance).


The port mapping isolator is for the case where ip per container is not
possible (due to ipam restriction, etc), but the user still wants to have
network namespace per container (for isolation, getting statistics, etc.)

Since all containers, even if they are in separate namespaces, share the
same IP, we have to use some other mechanism to tell which packet belongs
to which container. We use ports in that case. You can find more details
about port mapping isolator in this talk I gave in 2015 MesosCon:
https://www.youtube.com/watch?v=ZA96g1M4v8Y

- Jie

On Thu, Mar 30, 2017 at 2:13 AM, Thomas HUMMEL 
wrote:

>
> On 03/29/2017 07:25 PM, Jie Yu wrote:
>
> Thomas,
>
> I think you are confused about the port mapping for NAT purpose, and the port
> mapping isolator
> .
> Those two very different thing. The port mapping isolator (unfortunate
> naming), as described in the doc, gives you network namespace per container
> without requiring ip per container. No NAT is involved. I think for you
> case, you should not use it and it does not work for DockerContainerizer.
>
> Thanks,
>
> I'm not sure to understand what you say :
>
> - are you talking about the NAT feature of docker in BRIDGE mode ?
>
> - regarding the "port mapping isolator giving network namespace" : what
> confuses me is that, given the previous answers, I thought that in that
> case, the non-ephemeral port range was *shared* (as a ressource) between
> containers, which sounds to me at the opposite of the namespace concept (as
> a slightly different example 2 docker container have their own private 80
> port for instance).
>
> What am I missing ?
>
> Thanks
>
> --
> TH
>
>

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

On 03/29/2017 07:25 PM, Jie Yu wrote:

Thomas,

I think you are confused about the port mapping for NAT purpose, and 
the port mapping isolator 
. 
Those two very different thing. The port mapping isolator (unfortunate 
naming), as described in the doc, gives you network namespace per 
container without requiring ip per container. No NAT is involved. I 
think for you case, you should not use it and it does not work for 
DockerContainerizer.

Thanks,

I'm not sure to understand what you say :

- are you talking about the NAT feature of docker in BRIDGE mode ?

- regarding the "port mapping isolator giving network namespace" : what 
confuses me is that, given the previous answers, I thought that in that 
case, the non-ephemeral port range was *shared* (as a ressource) between 
containers, which sounds to me at the opposite of the namespace concept 
(as a slightly different example 2 docker container have their own 
private 80 port for instance).


What am I missing ?

Thanks

--
TH

Re: Mesos (and Marathon) port mapping

[Jie Yu]

Thomas,

I think you are confused about the port mapping for NAT purpose, and the port
mapping isolator
.
Those two very different thing. The port mapping isolator (unfortunate
naming), as described in the doc, gives you network namespace per container
without requiring ip per container. No NAT is involved. I think for you
case, you should not use it and it does not work for DockerContainerizer.

- Jie

On Wed, Mar 29, 2017 at 2:22 AM, Thomas HUMMEL 
wrote:

>
>
> On 03/28/2017 06:53 PM, Tomek Janiszewski wrote:
>
> 1. Mentioned port range is the Mesos Agent resource setting, so if you
> don't explicitly define port range it would be used.
> https://github.com/apache/mesos/blob/1.2.0/src/slave/constants.hpp#L86
>
> 2. With ports mapping two or more applications could attach to same
> container port but will be exposed under different host port.
>
>
> Thanks for your answer.
>
> 1. So it's not network/portmapping isolator specific, right ? Even without
> it, non-ephemeral ports would be considered as part of the offer and would
> be chosen in this range by default ?
>
> 2. So containers, even with network/port_mapping isolation, *share* the
> non-ephemeral port range, although doc states "The agent assigns each
> container a non-overlapping range of the ports" which I first read as "each
> container gets it's own port range", right ?
>
> So I am a bit confused since what's described here
>
> http://mesos.apache.org/documentation/latest/port-mapping-isolator/
>
> in the "Configuring network ports" seems to be valid even without port
> mapping isolator.
>
> Am I getting this right this time ?
>
> Thanks.
>
> --
> Thomas HUMMEL
>
>

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

On 03/29/2017 01:21 PM, Dick Davies wrote:

I should say this was tested around mesos 1.0, they may have changed
things - but yes this is vanilla networking, no CNI or anything like that.


As a matter of fact, that's what I experience.


But I'm guessing if you're using BRIDGE networking and specifying a
hostPort: you're causing work for yourself (unless you actually care what
port the slave is using).


Why would it make a difference regarding hostPort if you're using BRIDGE 
vs HOST mode ?


Still it doesn't explain why in Marathon UI I see a hostPort in the 
31000 - 32000 range when I specify a 9090 hostPort and I can verify on 
the slave that only 9090 is bound to.


Thanks

--
TH

Re: Mesos (and Marathon) port mapping

[Dick Davies]

I should say this was tested around mesos 1.0, they may have changed
things - but yes this is vanilla networking, no CNI or anything like that.

But I'm guessing if you're using BRIDGE networking and specifying a
hostPort: you're causing work for yourself (unless you actually care what
port the slave is using).

On 29 March 2017 at 10:22, Thomas HUMMEL  wrote:
>
>
> On 03/28/2017 06:53 PM, Tomek Janiszewski wrote:
>
> 1. Mentioned port range is the Mesos Agent resource setting, so if you don't
> explicitly define port range it would be used.
> https://github.com/apache/mesos/blob/1.2.0/src/slave/constants.hpp#L86
>
> 2. With ports mapping two or more applications could attach to same
> container port but will be exposed under different host port.
>
>
> Thanks for your answer.
>
> 1. So it's not network/portmapping isolator specific, right ? Even without
> it, non-ephemeral ports would be considered as part of the offer and would
> be chosen in this range by default ?
>
> 2. So containers, even with network/port_mapping isolation, *share* the
> non-ephemeral port range, although doc states "The agent assigns each
> container a non-overlapping range of the ports" which I first read as "each
> container gets it's own port range", right ?
>
> So I am a bit confused since what's described here
>
> http://mesos.apache.org/documentation/latest/port-mapping-isolator/
>
> in the "Configuring network ports" seems to be valid even without port
> mapping isolator.
>
> Am I getting this right this time ?
>
> Thanks.
>
> --
> Thomas HUMMEL
>

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

Also,


does network/portmapping isolator makes sense if the containerizer is 
docker ?

Re: Mesos (and Marathon) port mapping

[Thomas HUMMEL]

On 03/28/2017 06:53 PM, Tomek Janiszewski wrote:
1. Mentioned port range is the Mesos Agent resource setting, so if you 
don't explicitly define port range it would be used. 
https://github.com/apache/mesos/blob/1.2.0/src/slave/constants.hpp#L86


2. With ports mapping two or more applications could attach to same 
container port but will be exposed under different host port.




Thanks for your answer.

1. So it's not network/portmapping isolator specific, right ? Even 
without it, non-ephemeral ports would be considered as part of the offer 
and would be chosen in this range by default ?


2. So containers, even with network/port_mapping isolation, *share* the 
non-ephemeral port range, although doc states "The agent assigns each 
container a non-overlapping range of the ports" which I first read as 
"each container gets it's own port range", right ?


So I am a bit confused since what's described here

http://mesos.apache.org/documentation/latest/port-mapping-isolator/

in the "Configuring network ports" seems to be valid even without port 
mapping isolator.


Am I getting this right this time ?

Thanks.

--
Thomas HUMMEL

Re: Mesos (and Marathon) port mapping

[Dick Davies]

Try setting your hostPort to 0, to tell Mesos to select one
(which it will allocate out of the pool the mesos slave is set to use).

This works for me for redis:


{
  "container": {
"type": "DOCKER",
"docker": {
  "image": "redis",
  "network": "BRIDGE",
  "portMappings": [
{ "containerPort": 6379, "hostPort": 0, "protocol": "tcp"}
  ]
}
  },
  "ports": [0],
  "instances": 1,
  "cpus": 0.1,
  "mem": 128,
  "uris": []
}

(caveat: haven't run marathon or mesos for a little while)

On 28 March 2017 at 17:53, Tomek Janiszewski  wrote:
> 1. Mentioned port range is the Mesos Agent resource setting, so if you don't
> explicitly define port range it would be used.
> https://github.com/apache/mesos/blob/1.2.0/src/slave/constants.hpp#L86
>
> 2. With ports mapping two or more applications could attach to same
> container port but will be exposed under different host port.
>
> 3. I'm not sure if ports mappings works in Host mode. Try with require ports
> option enabled.
> https://github.com/mesosphere/marathon/blob/v1.3.9/docs/docs/ports.md
>
> 4. Yes, service ports are only for Marathon and don't propagate to Mesos.
> http://stackoverflow.com/a/39468348/1387612
>
>
> wt., 28.03.2017, 18:16 użytkownik Thomas HUMMEL 
> napisał:
>>
>> Hello,
>>
>> [Sorry if this post may seem more Marathon-oriented. It still contains
>> Mesos specific questions.]
>>
>> I'm in the process of discovering/testing/trying to understand Mesos and
>> Marathon.
>>
>> After having read some books and docs, I set up a small environment (9
>> linux
>> CentOS 7.3 VMs) consisting of :
>>
>>. 3 Mesos master - quorum = 2
>>. 3 Zookeepers servers running on the same host as the mesos servers
>>. 2 Mesos slaves
>>. 3 Marathon servers
>>. 1 HAproxy facing the Mesos servers
>>
>> Mesos has been installed from sources (1.2.0 version) and Marathon is
>> the 1.3.9
>> tarball comming from mesosphere
>>
>> I've deployed :
>>
>>. mesos-dns as a Marathon (not dockerized) application on one of the
>>  slaves (with a constraint) configured with my site DNS as resolvers
>> and only
>>  "host" as IPSources
>>
>>. marathon-lb as a Marathon dockerized app ("network": "HOST") with the
>>  simple (containerPort: 9090, hostPort: 9090, servicePort: 1)
>> portMapping,
>>  on the same slave using a constraint
>>
>> Everything works fine so far.
>> I've read :
>>
>>https://mesosphere.github.io/marathon/docs/ports.html
>>
>> and
>>
>>http://mesos.apache.org/documentation/latest/port-mapping-isolator/
>>
>> but I'm still quite confused by the following port-related questions :
>>
>> [Note : I'm not using "network/port_mapping" isolation for now. I sticked
>> to
>>
>>export MESOS_containerizers=docker,mesos]
>>
>> 1. for such a simple dockerized app :
>>
>> {
>>"id": "http-server",
>>"cmd": "python3 -m http.server 8080",
>>"cpus": 0.5,
>>"mem": 32.0,
>>"container": {
>>  "type": "DOCKER",
>>  "docker": {
>>"image": "python:3",
>>"network": "BRIDGE",
>>"portMappings": [
>>  { "containerPort": 8080, "hostPort": 31000, "servicePort": 5000 }
>>]
>>  }
>>},
>>"labels":{
>>  "HAPROXY_GROUP":"external"
>>}
>> }
>>
>> a) in HOST mode ("network": "HOST"), any hostPort seems to work (or at
>> least, let say 9090)
>>
>> b) in BRIDGE mode ("network": "BRIDGE"), the valid hostPort range seems
>> to be
>> [31000 - 32000], which seems to match the Mesos non-ephemeral port range
>> given
>> as en example in
>>
>>http://mesos.apache.org/documentation/latest/port-mapping-isolator/
>>
>> But I don't quite understand why since
>>
>>- I'm not using network/port_mapping isolation
>>- I didn't configured any port range anywhere in Mesos
>>
>> 2. Obviously in my setup, 2 apps on the same slave cannot have the same
>> hostPort. Would it be the same with network/port_mapping activated
>> since the
>> doc says : "he agent assigns each container a non-overlapping range
>> of the
>> ports"
>>
>> Am I correct assuming that a Marathon hostPort is to be understood
>> as taken among the non-ephemeral Mesos ports ?
>>
>> With network/port_mapping isolation, could 2 apps have the same
>> non-ephemeal port ? same question with ephemeral-port ? I doubt it but...
>> Is what is described in this doc valid for a dockerized container also
>> ?
>>
>> 3. the portMapping I configured for the dockerized ("network": "HOST")
>> marathon-lb app is
>>
>> "portMappings": [
>>{
>>  "containerPort": 9090,
>>  "hostPort": 9090,
>>  "servicePort": 1,
>>  "protocol": "tcp"
>>
>> on the slave I can verify :
>>
>># lsof -i :9090
>>COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
>>haproxy 29610 root6u  IPv4 461745  0t0  TCP *:websm (LISTEN)
>> But Marathon tells that my app is running on :
>>
>>mesos-slave1.it.pasteur.fr:31830
>>
>> I don't u

Re: Mesos (and Marathon) port mapping

[Tomek Janiszewski]

1. Mentioned port range is the Mesos Agent resource setting, so if you
don't explicitly define port range it would be used.
https://github.com/apache/mesos/blob/1.2.0/src/slave/constants.hpp#L86

2. With ports mapping two or more applications could attach to same
container port but will be exposed under different host port.

3. I'm not sure if ports mappings works in Host mode. Try with require
ports option enabled.
https://github.com/mesosphere/marathon/blob/v1.3.9/docs/docs/ports.md

4. Yes, service ports are only for Marathon and don't propagate to Mesos.
http://stackoverflow.com/a/39468348/1387612

wt., 28.03.2017, 18:16 użytkownik Thomas HUMMEL 
napisał:

Hello,

[Sorry if this post may seem more Marathon-oriented. It still contains
Mesos specific questions.]

I'm in the process of discovering/testing/trying to understand Mesos and
Marathon.

After having read some books and docs, I set up a small environment (9 linux
CentOS 7.3 VMs) consisting of :

   . 3 Mesos master - quorum = 2
   . 3 Zookeepers servers running on the same host as the mesos servers
   . 2 Mesos slaves
   . 3 Marathon servers
   . 1 HAproxy facing the Mesos servers

Mesos has been installed from sources (1.2.0 version) and Marathon is
the 1.3.9
tarball comming from mesosphere

I've deployed :

   . mesos-dns as a Marathon (not dockerized) application on one of the
 slaves (with a constraint) configured with my site DNS as resolvers
and only
 "host" as IPSources

   . marathon-lb as a Marathon dockerized app ("network": "HOST") with the
 simple (containerPort: 9090, hostPort: 9090, servicePort: 1)
portMapping,
 on the same slave using a constraint

Everything works fine so far.
I've read :

   https://mesosphere.github.io/marathon/docs/ports.html

and

   http://mesos.apache.org/documentation/latest/port-mapping-isolator/

but I'm still quite confused by the following port-related questions :

[Note : I'm not using "network/port_mapping" isolation for now. I sticked to

   export MESOS_containerizers=docker,mesos]

1. for such a simple dockerized app :

{
   "id": "http-server",
   "cmd": "python3 -m http.server 8080",
   "cpus": 0.5,
   "mem": 32.0,
   "container": {
 "type": "DOCKER",
 "docker": {
   "image": "python:3",
   "network": "BRIDGE",
   "portMappings": [
 { "containerPort": 8080, "hostPort": 31000, "servicePort": 5000 }
   ]
 }
   },
   "labels":{
 "HAPROXY_GROUP":"external"
   }
}

a) in HOST mode ("network": "HOST"), any hostPort seems to work (or at
least, let say 9090)

b) in BRIDGE mode ("network": "BRIDGE"), the valid hostPort range seems
to be
[31000 - 32000], which seems to match the Mesos non-ephemeral port range
given
as en example in

   http://mesos.apache.org/documentation/latest/port-mapping-isolator/

But I don't quite understand why since

   - I'm not using network/port_mapping isolation
   - I didn't configured any port range anywhere in Mesos

2. Obviously in my setup, 2 apps on the same slave cannot have the same
hostPort. Would it be the same with network/port_mapping activated
since the
doc says : "he agent assigns each container a non-overlapping range
of the
ports"

Am I correct assuming that a Marathon hostPort is to be understood
as taken among the non-ephemeral Mesos ports ?

With network/port_mapping isolation, could 2 apps have the same
non-ephemeal port ? same question with ephemeral-port ? I doubt it but...
Is what is described in this doc valid for a dockerized container also ?

3. the portMapping I configured for the dockerized ("network": "HOST")
marathon-lb app is

"portMappings": [
   {
 "containerPort": 9090,
 "hostPort": 9090,
 "servicePort": 1,
 "protocol": "tcp"

on the slave I can verify :

   # lsof -i :9090
   COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
   haproxy 29610 root6u  IPv4 461745  0t0  TCP *:websm (LISTEN)
But Marathon tells that my app is running on :

   mesos-slave1.it.pasteur.fr:31830

I don't understand where this port comes from, especially when I see
nobody's listening on it :

   lsof -i :31830

like if Marathon gave me a fake hostPort ?

4. My understanding is that Marathon service port are bound to only by apps
like marathon-lb. As a matter of fact, it doesn't seem to bother Mesos that
Marathon deploys 2 apps on the same slave with the same servicePort. Am
I correct ?

Thanks for your help

--
Thomas HUMMEL

Mesos (and Marathon) port mapping

[Thomas HUMMEL]

Hello,

[Sorry if this post may seem more Marathon-oriented. It still contains 
Mesos specific questions.]


I'm in the process of discovering/testing/trying to understand Mesos and 
Marathon.


After having read some books and docs, I set up a small environment (9 linux
CentOS 7.3 VMs) consisting of :

  . 3 Mesos master - quorum = 2
  . 3 Zookeepers servers running on the same host as the mesos servers
  . 2 Mesos slaves
  . 3 Marathon servers
  . 1 HAproxy facing the Mesos servers

Mesos has been installed from sources (1.2.0 version) and Marathon is 
the 1.3.9

tarball comming from mesosphere

I've deployed :

  . mesos-dns as a Marathon (not dockerized) application on one of the
slaves (with a constraint) configured with my site DNS as resolvers 
and only

"host" as IPSources

  . marathon-lb as a Marathon dockerized app ("network": "HOST") with the
simple (containerPort: 9090, hostPort: 9090, servicePort: 1) 
portMapping,

on the same slave using a constraint

Everything works fine so far.
I've read :

  https://mesosphere.github.io/marathon/docs/ports.html

and

  http://mesos.apache.org/documentation/latest/port-mapping-isolator/

but I'm still quite confused by the following port-related questions :

[Note : I'm not using "network/port_mapping" isolation for now. I sticked to

  export MESOS_containerizers=docker,mesos]

1. for such a simple dockerized app :

{
  "id": "http-server",
  "cmd": "python3 -m http.server 8080",
  "cpus": 0.5,
  "mem": 32.0,
  "container": {
"type": "DOCKER",
"docker": {
  "image": "python:3",
  "network": "BRIDGE",
  "portMappings": [
{ "containerPort": 8080, "hostPort": 31000, "servicePort": 5000 }
  ]
}
  },
  "labels":{
"HAPROXY_GROUP":"external"
  }
}

a) in HOST mode ("network": "HOST"), any hostPort seems to work (or at 
least, let say 9090)


b) in BRIDGE mode ("network": "BRIDGE"), the valid hostPort range seems 
to be
[31000 - 32000], which seems to match the Mesos non-ephemeral port range 
given

as en example in

  http://mesos.apache.org/documentation/latest/port-mapping-isolator/

But I don't quite understand why since

  - I'm not using network/port_mapping isolation
  - I didn't configured any port range anywhere in Mesos

2. Obviously in my setup, 2 apps on the same slave cannot have the same
   hostPort. Would it be the same with network/port_mapping activated 
since the
   doc says : "he agent assigns each container a non-overlapping range 
of the

   ports"

   Am I correct assuming that a Marathon hostPort is to be understood 
as taken among the non-ephemeral Mesos ports ?


   With network/port_mapping isolation, could 2 apps have the same 
non-ephemeal port ? same question with ephemeral-port ? I doubt it but...

   Is what is described in this doc valid for a dockerized container also ?

3. the portMapping I configured for the dockerized ("network": "HOST") 
marathon-lb app is


"portMappings": [
  {
"containerPort": 9090,
"hostPort": 9090,
"servicePort": 1,
"protocol": "tcp"

on the slave I can verify :

  # lsof -i :9090
  COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
  haproxy 29610 root6u  IPv4 461745  0t0  TCP *:websm (LISTEN)
But Marathon tells that my app is running on :

  mesos-slave1.it.pasteur.fr:31830

I don't understand where this port comes from, especially when I see 
nobody's listening on it :


  lsof -i :31830

like if Marathon gave me a fake hostPort ?

4. My understanding is that Marathon service port are bound to only by apps
like marathon-lb. As a matter of fact, it doesn't seem to bother Mesos that
Marathon deploys 2 apps on the same slave with the same servicePort. Am 
I correct ?


Thanks for your help

--
Thomas HUMMEL