Re: Mesos-docker-executor understanding

2017-07-21 Thread James Peach

> On Jul 19, 2017, at 10:05 AM, Thomas HUMMEL  wrote:
> 
> Hello,
> 
> I've read some books about Mesos, installed one multi-master cluster (for POC 
> purposes) with some frameworks (Marathon, Spark for instance) and watch some 
> talks.
> 
> Everything works and my understanding of Mesos is becoming clearer.
> However, I'm having a hard time fully understanding this section of the 
> documentation :
> 
>  http://mesos.apache.org/documentation/latest/containerizer-internals/
> 
> Note :
> 
> - my understanding is that Mesos is now heading towards a universal 
> containerizer, which is able to understand docker (among others) images thus 
> beeing able to "do some docker" without a docker daemon.


> 
> - Also, I don't think Mesos supports nester containers yet either.

http://mesos.apache.org/documentation/latest/nested-container-and-task-group/

> 
> But I'm not really sure about what is the mesos-docker-executor, as opposed 
> to mesos-executor.
> 
> - For instance, in the "A)" case of the slave running in a container, the doc 
> states :
> 
> "if the task does not include an executor i.e. it defines a command, the 
> default executor mesos-docker-executor is launched in a docker container to 
> execute the command via Docker CLI."
> 
> Why Docker CLI ? We are not in a shell context, are we ?
> Also, will the command be launched in another docker container, different 
> from the one running mesos-docker-executor ?
> 
> - In the B) case where the slave is not running in a container, the doc 
> states :
> 
> "If task does not include an executor i.e. it defines a command, a subprocess 
> is forked to execute the default executor mesos-docker-executor. 
> mesos-docker-executor then spawns a shell to execute the command via Docker 
> CLI."
> 
> Why is the mesos-docker-executor not run in a container ? Also, why does it 
> not use docker API directly ?
> 
> Can you help me figuring out how exactly mesos-docker-executor works and what 
> its specificality relative to the mesos-executor ?

From the scheduler's POV, I'd say you don't really need to know the difference. 
In the API you need to specify the container type to switch between using the 
Docker containerized and the Mesos containerizer. For example, you can 
experiment by passing JSON to the --task option of mesos-execute:

{
  "name": "sleep",
  "agent_id": { "value": "any" },
  "task_id": {
"value": "some-unique-uuid"
  },
  "resources": [
{
  "name": "cpus",
  "type": "SCALAR",
  "scalar": {
"value": 0.4
  },
  "role": "*"
},
{
  "name": "mem",
  "type": "SCALAR",
  "scalar": {
"value": 32
  },
  "role": "*"
}
  ],
  "command": {
"value": "command-line that you want to run",
"environment": {
" variables": [
{ "name": "GLOG_v", "value": "2" }
]
}
  },
  "container": {
"type": "MESOS",
"mesos": {
  "image": {
"type": "DOCKER",
"docker": {
"name": "image/name"
}
  }
}
  }
}



Mesos-docker-executor understanding

2017-07-19 Thread Thomas HUMMEL

Hello,

I've read some books about Mesos, installed one multi-master cluster 
(for POC purposes) with some frameworks (Marathon, Spark for instance) 
and watch some talks.


Everything works and my understanding of Mesos is becoming clearer.
However, I'm having a hard time fully understanding this section of the 
documentation :


  http://mesos.apache.org/documentation/latest/containerizer-internals/

Note :

- my understanding is that Mesos is now heading towards a universal 
containerizer, which is able to understand docker (among others) images 
thus beeing able to "do some docker" without a docker daemon.


- Also, I don't think Mesos supports nester containers yet either.

But I'm not really sure about what is the mesos-docker-executor, as 
opposed to mesos-executor.


- For instance, in the "A)" case of the slave running in a container, 
the doc states :


"if the task does not include an executor i.e. it defines a command, the 
default executor mesos-docker-executor is launched in a docker container 
to execute the command via Docker CLI."


Why Docker CLI ? We are not in a shell context, are we ?
Also, will the command be launched in another docker container, 
different from the one running mesos-docker-executor ?


- In the B) case where the slave is not running in a container, the doc 
states :


"If task does not include an executor i.e. it defines a command, a 
subprocess is forked to execute the default executor 
mesos-docker-executor. mesos-docker-executor then spawns a shell to 
execute the command via Docker CLI."


Why is the mesos-docker-executor not run in a container ? Also, why does 
it not use docker API directly ?


Can you help me figuring out how exactly mesos-docker-executor works and 
what its specificality relative to the mesos-executor ?


Thanks

--
Thomas HUMMEL