Re: Deprecate MESOS_DIRECTORY executor environment variable

[Vinod Kone]

If the sandbox host path is not yet available in v1 API, it should be made
available. I'll be happy to shepherd.

The rules for APIs stipulate that the deprecation period doesn't start
> until the next major release (2.0).
>

We need to revise the API compatibility rules a bit. Right now the rules
for API conflict a bit with rules of deprecation in the versioning doc.
It's my bad because I wrote those in different points of time but didn't
reconcile them.

[Port mapping] Discuss where to validate the host port container wants to expose to is from the offered resources

[Qian Zhang]

Hi All,

I am currently working on MESOS-6106
, the purpose of this
ticket is to validate the host port container wants to expose to is from
the offered resources, so that we can ensure container will not expose to
an arbitrary host port.

My idea is, we validate `ContainerInfo.NetworkInfo.PortMapping.host_port`
against the offered resources in master ("Master::_accept()") by adding a
new validate method in "src/master/validation.cpp", and in agent, we can
also double check it in the "prepare()" method of CNI isolator.

However, I see for DockContainerizer, we do the similar validation on in
agent (
https://github.com/apache/mesos/blob/1.0.1/src/docker/docker.cpp#L718:#L729)
but not in master. @Tim Chen, can you please let me know why we did such
validation only in agent but not in master? Is it because
"ContainerInfo.DockerInfo.PortMapping" is DockerContainerizer specific and
we only want to do generic validation in master?

@Jie and @BenM, please also let me know your comments, thanks!


Thanks,
Qian Zhang

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Yan Xu]

The rules for APIs stipulate that the deprecation period doesn't start
until the next major release (2.0). I suppose environment variable changes
don't qualify (or do they? This is how the agent interface with the
executor and requires executor developers to change their programs). So
when should it start (1.1 vs. 2.0)?

On Fri, Oct 7, 2016 at 5:33 PM, Jie Yu  wrote:

> https://github.com/apache/mesos/blob/master/docs/versioning.md
>
> "The deprecation period for any given feature will be 6 months. Having a
> set period allows Mesos developers to not indefinitely accrue technical
> debt and allows users time to plan for upgrades."
>
> - Jie
>
> On Fri, Oct 7, 2016 at 5:28 PM, Zameer Manji  wrote:
>
> > Jie,
> >
> > Without commenting on this deprecation, how is this going to work now
> that
> > Mesos is 1.0?
> >
> > What is the definition of "deprecate" being used here? Is it something
> that
> > will be removed in Mesos 2.0?
> >
> > On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:
> >
> > > Hi,
> > >
> > > Want to initiate a discussion here. Before Mesos containerizer has
> > > container image support (all containers share the same host file
> system),
> > > $MESOS_DIRECTORY env variable is used to let executor know their
> sandbox
> > > location.
> > >
> > > Later, we introduced container image support to Mesos containerizer so
> > that
> > > each container can has its own root filesystem. Due to some historical
> > > reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to
> > the
> > > sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even
> > if
> > > the container has its own root filesystem. And introduced a new
> > > $MESOS_SANDBOX to point to the sandbox in the container's root
> filesystem
> > > (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> > > filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
> > >
> > > Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> > > confusing to executor writers, and it'll be an error if they try to
> > access
> > > $MESOS_DIRECTORY if their container has a root filesystem defined.
> > >
> > > - Jie
> > >
> > > --
> > > Zameer Manji
> > >
> >
>

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Jie Yu]

>
> This sandbox host path information is available through the agent's
> /state endpoint but I don't think it's available via the agent HTTP API?


@vinodkone, can you confirm this?

On Fri, Oct 7, 2016 at 6:10 PM, Yan Xu  wrote:

> I agree that the executor shouldn't need to known about the sandbox host
> path for itself but today it provides a way for the information to be
> propagated to the scheduler or external systems.
>
> I don't remember how thermos observer works but I guess the executor
> registers itself to a the observer and provides it with information about
> the host path for its sandbox? I could be incorrect about thermos but this
> usage pattern exists elsewhere as well and will break after this change.
>
> Overall I agree that we should deprecate this usage pattern but it's worth
> providing solutions to replace it. This sandbox host path information is
> available through the agent's /state endpoint but I don't think it's
> available via the agent HTTP API? The `GetExecutors` call doesn't return
> this path but its `ListFiles` and `ReadFile` calls expect this path. Should
> the host path be added to the result of GetExecutors call? (This being just
> one example)
>
> On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:
>
> > Hi,
> >
> > Want to initiate a discussion here. Before Mesos containerizer has
> > container image support (all containers share the same host file system),
> > $MESOS_DIRECTORY env variable is used to let executor know their sandbox
> > location.
> >
> > Later, we introduced container image support to Mesos containerizer so
> that
> > each container can has its own root filesystem. Due to some historical
> > reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to
> the
> > sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even
> if
> > the container has its own root filesystem. And introduced a new
> > $MESOS_SANDBOX to point to the sandbox in the container's root filesystem
> > (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> > filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
> >
> > Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> > confusing to executor writers, and it'll be an error if they try to
> access
> > $MESOS_DIRECTORY if their container has a root filesystem defined.
> >
> > - Jie
> >
>

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Yan Xu]

I agree that the executor shouldn't need to known about the sandbox host
path for itself but today it provides a way for the information to be
propagated to the scheduler or external systems.

I don't remember how thermos observer works but I guess the executor
registers itself to a the observer and provides it with information about
the host path for its sandbox? I could be incorrect about thermos but this
usage pattern exists elsewhere as well and will break after this change.

Overall I agree that we should deprecate this usage pattern but it's worth
providing solutions to replace it. This sandbox host path information is
available through the agent's /state endpoint but I don't think it's
available via the agent HTTP API? The `GetExecutors` call doesn't return
this path but its `ListFiles` and `ReadFile` calls expect this path. Should
the host path be added to the result of GetExecutors call? (This being just
one example)

On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:

> Hi,
>
> Want to initiate a discussion here. Before Mesos containerizer has
> container image support (all containers share the same host file system),
> $MESOS_DIRECTORY env variable is used to let executor know their sandbox
> location.
>
> Later, we introduced container image support to Mesos containerizer so that
> each container can has its own root filesystem. Due to some historical
> reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to the
> sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even if
> the container has its own root filesystem. And introduced a new
> $MESOS_SANDBOX to point to the sandbox in the container's root filesystem
> (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
>
> Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> confusing to executor writers, and it'll be an error if they try to access
> $MESOS_DIRECTORY if their container has a root filesystem defined.
>
> - Jie
>

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Zameer Manji]

Thanks for clearing that up.

I've cced Joshua Cohen (who is currently maintaining thermos) to weigh in
here.

I suspect that deprecating this blindly might break something for tasks
under the Docker Containerizer.

On Fri, Oct 7, 2016 at 5:33 PM, Jie Yu  wrote:

> https://github.com/apache/mesos/blob/master/docs/versioning.md
>
> "The deprecation period for any given feature will be 6 months. Having a
> set period allows Mesos developers to not indefinitely accrue technical
> debt and allows users time to plan for upgrades."
>
> - Jie
>
> On Fri, Oct 7, 2016 at 5:28 PM, Zameer Manji  wrote:
>
> > Jie,
> >
> > Without commenting on this deprecation, how is this going to work now
> that
> > Mesos is 1.0?
> >
> > What is the definition of "deprecate" being used here? Is it something
> that
> > will be removed in Mesos 2.0?
> >
> > On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:
> >
> > > Hi,
> > >
> > > Want to initiate a discussion here. Before Mesos containerizer has
> > > container image support (all containers share the same host file
> system),
> > > $MESOS_DIRECTORY env variable is used to let executor know their
> sandbox
> > > location.
> > >
> > > Later, we introduced container image support to Mesos containerizer so
> > that
> > > each container can has its own root filesystem. Due to some historical
> > > reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to
> > the
> > > sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even
> > if
> > > the container has its own root filesystem. And introduced a new
> > > $MESOS_SANDBOX to point to the sandbox in the container's root
> filesystem
> > > (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> > > filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
> > >
> > > Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> > > confusing to executor writers, and it'll be an error if they try to
> > access
> > > $MESOS_DIRECTORY if their container has a root filesystem defined.
> > >
> > > - Jie
> > >
> > > --
> > > Zameer Manji
> > >
> >
>
> --
> Zameer Manji
>

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Jie Yu]

https://github.com/apache/mesos/blob/master/docs/versioning.md

"The deprecation period for any given feature will be 6 months. Having a
set period allows Mesos developers to not indefinitely accrue technical
debt and allows users time to plan for upgrades."

- Jie

On Fri, Oct 7, 2016 at 5:28 PM, Zameer Manji  wrote:

> Jie,
>
> Without commenting on this deprecation, how is this going to work now that
> Mesos is 1.0?
>
> What is the definition of "deprecate" being used here? Is it something that
> will be removed in Mesos 2.0?
>
> On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:
>
> > Hi,
> >
> > Want to initiate a discussion here. Before Mesos containerizer has
> > container image support (all containers share the same host file system),
> > $MESOS_DIRECTORY env variable is used to let executor know their sandbox
> > location.
> >
> > Later, we introduced container image support to Mesos containerizer so
> that
> > each container can has its own root filesystem. Due to some historical
> > reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to
> the
> > sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even
> if
> > the container has its own root filesystem. And introduced a new
> > $MESOS_SANDBOX to point to the sandbox in the container's root filesystem
> > (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> > filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
> >
> > Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> > confusing to executor writers, and it'll be an error if they try to
> access
> > $MESOS_DIRECTORY if their container has a root filesystem defined.
> >
> > - Jie
> >
> > --
> > Zameer Manji
> >
>

Re: Deprecate MESOS_DIRECTORY executor environment variable

[Zameer Manji]

Jie,

Without commenting on this deprecation, how is this going to work now that
Mesos is 1.0?

What is the definition of "deprecate" being used here? Is it something that
will be removed in Mesos 2.0?

On Fri, Oct 7, 2016 at 4:49 PM, Jie Yu  wrote:

> Hi,
>
> Want to initiate a discussion here. Before Mesos containerizer has
> container image support (all containers share the same host file system),
> $MESOS_DIRECTORY env variable is used to let executor know their sandbox
> location.
>
> Later, we introduced container image support to Mesos containerizer so that
> each container can has its own root filesystem. Due to some historical
> reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to the
> sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even if
> the container has its own root filesystem. And introduced a new
> $MESOS_SANDBOX to point to the sandbox in the container's root filesystem
> (e.g., `/mnt/mesos/sandbox`). If the container does not have a root
> filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.
>
> Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
> confusing to executor writers, and it'll be an error if they try to access
> $MESOS_DIRECTORY if their container has a root filesystem defined.
>
> - Jie
>
> --
> Zameer Manji
>

Deprecate MESOS_DIRECTORY executor environment variable

[Jie Yu]

Hi,

Want to initiate a discussion here. Before Mesos containerizer has
container image support (all containers share the same host file system),
$MESOS_DIRECTORY env variable is used to let executor know their sandbox
location.

Later, we introduced container image support to Mesos containerizer so that
each container can has its own root filesystem. Due to some historical
reason (thermos), we decided to keep $MESOS_DIRECTORY to be the path to the
sandbox on the host filesystem (e.g., `/var/lib/mesos/slaves/...`) even if
the container has its own root filesystem. And introduced a new
$MESOS_SANDBOX to point to the sandbox in the container's root filesystem
(e.g., `/mnt/mesos/sandbox`). If the container does not have a root
filesystem, $MESOS_DIRECTORY == $MESOS_SANDBOX.

Now, we plan to deprecate $MESOS_DIRECTORY because it'll be really
confusing to executor writers, and it'll be an error if they try to access
$MESOS_DIRECTORY if their container has a root filesystem defined.

- Jie

Re: 1.1.0 release

[Vinod Kone]

I think you need to clean up the JIRA a bit.

1) Make sure unresolved tickets do not have fix version (1.1.0) set.
2) Move "Fix version 1.1.0" to "Target version 1.1.0".

2) might obviate the need for 1).



On Fri, Oct 7, 2016 at 7:24 AM, Till Toenshoff  wrote:

> Hi everyone!
>
> its us who will be the Release Managers for 1.1.0 - Alex and Till!
>
> We are planning to cut the next release (1.1.0) within three workdays -
> that would be Wednesday next week. So, if you have any patches that need to
> get into 1.1.0 make sure that either is already in the master branch or the
> corresponding ticket has a target version set to 1.1.0.
>
> The release dashboard:
> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329720
>
> Alex & Till
>

1.1.0 release

[Till Toenshoff]

Hi everyone!

its us who will be the Release Managers for 1.1.0 - Alex and Till! 

We are planning to cut the next release (1.1.0) within three workdays - that 
would be Wednesday next week. So, if you have any patches that need to get into 
1.1.0 make sure that either is already in the master branch or the 
corresponding ticket has a target version set to 1.1.0.

The release dashboard:
https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329720 


Alex & Till

Re: LXC Container Support

[Tom Barber]

Everyone appears asleep on IRC so for a non real time response:

If I were looking to replicate the docker calls but for LXC, I'd be
updating stuff in the containerizer package right? Not the top level appc
and docker packages.
https://github.com/apache/mesos/blob/master/src/slave/containerizer/

Thanks

Tom

On Thu, Oct 6, 2016 at 10:59 PM, Tom Barber  wrote:

> Also just as an aside, being able to run system containers and not
> precanned Docker applications would seem to certainly be a missing feature
> that Mesos could benefit from, as it would allow sysadmins to spin up
> vanilla Ubuntu, Centos, Debian etc images without doing something funky
> like using the Docker image and tailing /dev/null. :)
>
> On Thu, Oct 6, 2016 at 2:11 AM, Tom Barber 
> wrote:
>
>> Thanks guys.  I do a lot of work with the juju ecosystem and lxc
>> containers
>> to deploy workloads with that, for example multi node big top clusters
>> etc.
>> It's far more flexible using LXC system containers than pre configured app
>> containers. This works on cloud and bare metal but I'd also like to be
>> able
>> to deploy containers to Mesos using the same technology stack.
>>
>> Cheers
>> Tom
>>
>> On 6 Oct 2016 00:51, "Jie Yu"  wrote:
>>
>> > Tom,
>> >
>> > I'm looking to add LXC/LXD support, POC initially then hopefully bake it
>> > > into some Mesos libraries. From what I can tell Mesos doesn't have
>> hooks
>> > > for LXC in the new container setup and also the External Shim that was
>> > > underdevelopment is Deprecated. Sound roughly accurate so far?
>> >
>> >
>> > Yup. External containerizer has been deprecated and removed. The
>> direction
>> > is to maintain only one containerizer in the future, and make that
>> > containerizer better and supports various containerization requests.
>> >
>> > If so, would extending the Unified Containerizer be the correct way to
>> go
>> > > about adding LXC/LXD support?
>> >
>> >
>> > Yes, I would prefer exploring this direction. But even before that, it
>> > would be nice to understand the motivation for using lxd. I'd prefer we
>> > find out the feature gap and augment unified containerizer to fill the
>> gap.
>> >
>> > - Jie
>> >
>> > On Wed, Oct 5, 2016 at 3:10 PM, Tom Barber 
>> > wrote:
>> >
>> > > Hello folks,
>> > >
>> > > I've been Googling Mesos container support a bit recently, I see
>> support
>> > > for Mesos(AppC) and Docker in the latest versions.
>> > >
>> > > I'm looking to add LXC/LXD support, POC initially then hopefully bake
>> it
>> > > into some Mesos libraries. From what I can tell Mesos doesn't have
>> hooks
>> > > for LXC in the new container setup and also the External Shim that was
>> > > underdevelopment is Deprecated. Sound roughly accurate so far?
>> > >
>> > > If so, would extending the Unified Containerizer be the correct way
>> to go
>> > > about adding LXC/LXD support? (
>> > > http://mesos.apache.org/documentation/latest/container-image/)
>> > >
>> > > Thanks
>> > >
>> > > Tom
>> > >
>> >
>>
>
>