A port mapping plugin for CNI

[Avinash Sridharan]

Hi All,
 One of the most used networking mode when users run containers using the
`DockerContainerizer` is docker bridge mode networking. In this mode
containers run in an isolator private address space, and services on the
container are exposed to the outside world using DNAT.

As we move towards the unified containerizer becoming the de-facto
container runtime, and users start running their docker containers on the
`MesosContainerizer`, the expectation of exposing services running on
isolator bridges with DNAT starts becoming a must have.

With the introduction of the `network/cni` isolator we can use CNI plugins
to start attaching containers on the `MesosContainerizer` to different
types of IP networks. Corresponding to docker bridge network, CNI has its
own bridge plugin, however unlike docker bridge networking the CNI bridge
plugin does not provide DNAT services to expose containers on a bridge.
None of the core CNI plugins provide a port mapping functionality, and it
is only recently that there seems to be a push for having port mapping
functionality in a CNI plugin.

We are therefore proposing implementing a CNI plugin that can setup port
mapping rules for different CNI plugins for Mesos. This CNI plugin is
generic enough that it can be used in conjunction with any other CNI
plugin, such as the bridge plugin.

The motivation, design and operational aspects of the plugin have been
captured in this document:
https://docs.google.com/document/d/1ZwXZ_utpxmy9vccYiL0q86efgpWpjmmKLQ0S4Mmz9N4/edit?usp=sharing

Would be great if the community can share their feedback on the proposed
port mapping CNI plugin.

Thanks,
-- 
Avinash Sridharan, Mesosphere
+1 (323) 702 5245

Re: Shepherd for MESOS-5929

[Benjamin Mahler]

I pushed your patch, thanks!

On Fri, Aug 5, 2016 at 12:28 PM, Charles Allen <
charles.al...@metamarkets.com> wrote:

> https://reviews.apache.org/r/50862/ which is a proposed fix for MESOS-5929
> needs a shepherd. Is anyone able to do that?
>

Shepherd for MESOS-5929

[Charles Allen]

https://reviews.apache.org/r/50862/ which is a proposed fix for MESOS-5929
needs a shepherd. Is anyone able to do that?

Re: Protobuf long number JSON serialisation

[Tomek Janiszewski]

Created a patch https://reviews.apache.org/r/50851/
Looking for shepherd.

pt., 5.08.2016 o 09:49 użytkownik Tomek Janiszewski 
napisał:

> I reported issue for this https://issues.apache.org/jira/browse/MESOS-5995
>
> pt., 5.08.2016 o 02:35 użytkownik Joseph Wu 
> napisał:
>
>> This is not necessarily a bug, but I think we can safely extend our
>> parsing
>> code to handle this case.
>>
>> This is the method that would need to change:
>>
>> https://github.com/apache/mesos/blob/e859d3ae8d8ff7349327b9e6a89edd6f98d2b7a1/3rdparty/stout/include/stout/protobuf.hpp#L433-L435
>>
>> On Thu, Aug 4, 2016 at 4:04 PM, Anand Mazumdar 
>> wrote:
>>
>> > Tomek,
>> >
>> > Thanks for reporting this. Looks like a bug in our JSON -> Protobuf
>> > parsing code. Mind filing a JIRA issue?
>> >
>> > -anand
>> >
>> >
>> > > On Aug 4, 2016, at 2:04 PM, Tomek Janiszewski 
>> wrote:
>> > >
>> > > Hi
>> > >
>> > > I have a problem with HTTP API. Proto2 does not specify JSON mappings
>> but
>> > > Proto3 does and it recommend to map 64bit numbers as a string.
>> > > Unfortunately Mesos does not accepts strings in places of uint64 and
>> > return
>> > > 400 Bad Request error Failed to convert JSON into Call protobuf: Not
>> > > expecting a JSON string for field 'value'.
>> > > Is this by purpose or is this a bug?
>> > >
>> > > Best
>> > > Tomek
>> >
>> >
>>
>

Re: On creating actor instances in Mesos

[Gastón Kleiman]

Hi,

As you can see in MESOS-5917, I made sure that all the processes I could
find specify a distinguishable IDs.

After talking to Alex R and Ben M, I enforced the following naming schema:

For processes created by Mesos:

   - Every process must specify a distinguishable process ID.

   - The process ID must contain only lowercase characters and use a dash
as word separator, e.g., `zookeeper-leader-contender`.

   - A common pattern is to prefix the ID with the name of the component,
e.g., `log-recover`, `logrotate-logger`.

For processes created by libprocess:

   - The ID of a process internal to libprocess (one with not associated
route/endpoint) must be enclosed in double underscores, and use underscores
instead of dashes as word separator, e.g., `__authentication_router__`.

   - The Mesos process ID naming schema applies to libprocess exposed to
libprocess users, via a route/associated to an endpoint, e.g., use `help`
instead of `__help__`.

>From now on, please make sure you adhere to this naming scheme when
defining new processes!

Thanks,

-Gastón


On Sat, Jul 23, 2016 at 12:04 AM, Timothy Chen  wrote:

> +1
>
> Previously when I was adding tracing to Mesos this was one of the most
> missing piece of metadata to easily detect sender and receiver
> information.
>
> Tim
>
> On Fri, Jul 22, 2016 at 11:07 PM, Alex Rukletsov 
> wrote:
> > Folks,
> >
> > I've noticed recently that some actors do not specify a distinguishable
> > actor ID. As a result, it may be hard to match output to a specific
> actor,
> > for example an excerpt from "__processes__" endpoint:
> >
> > [{"events":[],"id":"(10)"},{"events":[],"id":"(11)"},{"
> events":[],"id":"(1279859)"},{"events":[],"id":"(15)"},
> > ... ]
> >
> > Every time you create an actor, i.e., a `ProcessBase` instance, you most
> > probably want to give it a meaningful id. Consider `StatusUpdateManager`.
> > Currently, the code does not specify the ID:
> >
> > StatusUpdateManagerProcess::StatusUpdateManagerProcess(const Flags&
> _flags)
> >   : flags(_flags), paused(false) {}
> >
> > Instead, a preferred way of calling the `StatusUpdateManagerProcess`
> c-tor
> > would be:
> >
> > StatusUpdateManagerProcess::StatusUpdateManagerProcess(const Flags&
> _flags)
> >   : ProcessBase(process::ID::generate("status-update-manager")),
> > flags(_flags),
> > paused(false) {}
> >
> > Best,
> > Alex
>

Re: Protobuf long number JSON serialisation

[Tomek Janiszewski]

I reported issue for this https://issues.apache.org/jira/browse/MESOS-5995

pt., 5.08.2016 o 02:35 użytkownik Joseph Wu  napisał:

> This is not necessarily a bug, but I think we can safely extend our parsing
> code to handle this case.
>
> This is the method that would need to change:
>
> https://github.com/apache/mesos/blob/e859d3ae8d8ff7349327b9e6a89edd6f98d2b7a1/3rdparty/stout/include/stout/protobuf.hpp#L433-L435
>
> On Thu, Aug 4, 2016 at 4:04 PM, Anand Mazumdar 
> wrote:
>
> > Tomek,
> >
> > Thanks for reporting this. Looks like a bug in our JSON -> Protobuf
> > parsing code. Mind filing a JIRA issue?
> >
> > -anand
> >
> >
> > > On Aug 4, 2016, at 2:04 PM, Tomek Janiszewski 
> wrote:
> > >
> > > Hi
> > >
> > > I have a problem with HTTP API. Proto2 does not specify JSON mappings
> but
> > > Proto3 does and it recommend to map 64bit numbers as a string.
> > > Unfortunately Mesos does not accepts strings in places of uint64 and
> > return
> > > 400 Bad Request error Failed to convert JSON into Call protobuf: Not
> > > expecting a JSON string for field 'value'.
> > > Is this by purpose or is this a bug?
> > >
> > > Best
> > > Tomek
> >
> >
>