Re: How to launch a framework via Mesos REST API
Thanks so much for the clarification, Benjamin. Wenzhao On Fri, Jul 21, 2017 at 11:22 AM, Benjamin Bannier < benjamin.bann...@mesosphere.io> wrote: > Hi Wenzhao, > > > I agree your statement, Marathon is only a management layer over Mesos. > > I believe I did not write that. > > > But, we can run a framework on Mesos via invoking Marathon's REST API. > > How does this happen? I think Marathon internally calls Mesos's API to > start the framework. But I don't see Mesos REST API to create a > framework. I only know to run a framework via "mesos-execute" command. > > > > Or maybe Marathon implements its own scheduler, and execute a framework > by itself? > > You are still mixing a few things up: > > * Marathon is a Mesos framework. A framework primarily consists of a > scheduler. > * `mesos-execute` creates a one-off framework. That framework will submit > and manage the given task(s). The framework is stopped when the task(s) > finish. > * There are different APIs. One of them is the scheduler API which > frameworks use to talk to Mesos, another one the operator API which can be > use to administrate Mesos. One cannot start tasks with the operator API. > > The documentation contains a lot of this, including diagrams making things > clearer, > > https://mesos.apache.org/documentation/ > > I’d especially encourage you to read the architecture overview, > > https://mesos.apache.org/documentation/latest/architecture/ > > and the documentation of the operator HTTP API, > > https://mesos.apache.org/documentation/latest/operator-http-api/ > > > Cheers and HTH, > > Benjamin
Re: Mesos-docker-executor understanding
> On Jul 19, 2017, at 10:05 AM, Thomas HUMMEL wrote:
>
> Hello,
>
> I've read some books about Mesos, installed one multi-master cluster (for POC
> purposes) with some frameworks (Marathon, Spark for instance) and watch some
> talks.
>
> Everything works and my understanding of Mesos is becoming clearer.
> However, I'm having a hard time fully understanding this section of the
> documentation :
>
> http://mesos.apache.org/documentation/latest/containerizer-internals/
>
> Note :
>
> - my understanding is that Mesos is now heading towards a universal
> containerizer, which is able to understand docker (among others) images thus
> beeing able to "do some docker" without a docker daemon.
>
> - Also, I don't think Mesos supports nester containers yet either.
http://mesos.apache.org/documentation/latest/nested-container-and-task-group/
>
> But I'm not really sure about what is the mesos-docker-executor, as opposed
> to mesos-executor.
>
> - For instance, in the "A)" case of the slave running in a container, the doc
> states :
>
> "if the task does not include an executor i.e. it defines a command, the
> default executor mesos-docker-executor is launched in a docker container to
> execute the command via Docker CLI."
>
> Why Docker CLI ? We are not in a shell context, are we ?
> Also, will the command be launched in another docker container, different
> from the one running mesos-docker-executor ?
>
> - In the B) case where the slave is not running in a container, the doc
> states :
>
> "If task does not include an executor i.e. it defines a command, a subprocess
> is forked to execute the default executor mesos-docker-executor.
> mesos-docker-executor then spawns a shell to execute the command via Docker
> CLI."
>
> Why is the mesos-docker-executor not run in a container ? Also, why does it
> not use docker API directly ?
>
> Can you help me figuring out how exactly mesos-docker-executor works and what
> its specificality relative to the mesos-executor ?
From the scheduler's POV, I'd say you don't really need to know the difference.
In the API you need to specify the container type to switch between using the
Docker containerized and the Mesos containerizer. For example, you can
experiment by passing JSON to the --task option of mesos-execute:
{
"name": "sleep",
"agent_id": { "value": "any" },
"task_id": {
"value": "some-unique-uuid"
},
"resources": [
{
"name": "cpus",
"type": "SCALAR",
"scalar": {
"value": 0.4
},
"role": "*"
},
{
"name": "mem",
"type": "SCALAR",
"scalar": {
"value": 32
},
"role": "*"
}
],
"command": {
"value": "command-line that you want to run",
"environment": {
" variables": [
{ "name": "GLOG_v", "value": "2" }
]
}
},
"container": {
"type": "MESOS",
"mesos": {
"image": {
"type": "DOCKER",
"docker": {
"name": "image/name"
}
}
}
}
}
Re: How to launch a framework via Mesos REST API
Hi Wenzhao, > I agree your statement, Marathon is only a management layer over Mesos. I believe I did not write that. > But, we can run a framework on Mesos via invoking Marathon's REST API. > How does this happen? I think Marathon internally calls Mesos's API to start > the framework. But I don't see Mesos REST API to create a framework. I only > know to run a framework via "mesos-execute" command. > > Or maybe Marathon implements its own scheduler, and execute a framework by > itself? You are still mixing a few things up: * Marathon is a Mesos framework. A framework primarily consists of a scheduler. * `mesos-execute` creates a one-off framework. That framework will submit and manage the given task(s). The framework is stopped when the task(s) finish. * There are different APIs. One of them is the scheduler API which frameworks use to talk to Mesos, another one the operator API which can be use to administrate Mesos. One cannot start tasks with the operator API. The documentation contains a lot of this, including diagrams making things clearer, https://mesos.apache.org/documentation/ I’d especially encourage you to read the architecture overview, https://mesos.apache.org/documentation/latest/architecture/ and the documentation of the operator HTTP API, https://mesos.apache.org/documentation/latest/operator-http-api/ Cheers and HTH, Benjamin
Re: How to launch a framework via Mesos REST API
Hi, Benjamin: Thanks for the feedback. I agree your statement, Marathon is only a management layer over Mesos. But, we can run a framework on Mesos via invoking Marathon's REST API. How does this happen? I think Marathon internally calls Mesos's API to start the framework. But I don't see Mesos REST API to create a framework. I only know to run a framework via "mesos-execute" command. Or maybe Marathon implements its own scheduler, and execute a framework by itself? Thanks Wenzhao On Fri, Jul 21, 2017 at 8:28 AM, Benjamin Bannier < benjamin.bann...@mesosphere.io> wrote: > Hi Wenzhao, > > I am not sure I understand you exactly. > > All Mesos tasks (the stuff running on and managed by Mesos agents) always > belong to a framework; this framework launches and supervises them, and > manages their lifetime. It is not possible to start tasks without a > framework. > > Marathon is a framework. Usually the framework process is not started as a > Mesos task, but instead started by e.g., init.d/systemd, so the framework > itself is not supervised by Mesos. Users can start Mesos tasks by e.g., > hitting Marathon endpoints. Marathon then starts a task with Mesos and > makes sure that the task keeps running (similar to how e.g., systemd might > make sure that the Marathon framework process is running). > > Frameworks can talk to Mesos with two different APIs (often called v0 and > v1); v0 is a C++ API while v1 communicates over HTTP. v1 is not a REST API. > > To launch Mesos tasks you will need a framework. You can either write a > framework yourself, or use an existing one (there’s one list of some > frameworks here, https://mesos.apache.org/documentation/latest/frameworks/ > ). > > > Cheers, > > Benjamin
Re: How to launch a framework via Mesos REST API
Hi Wenzhao, I am not sure I understand you exactly. All Mesos tasks (the stuff running on and managed by Mesos agents) always belong to a framework; this framework launches and supervises them, and manages their lifetime. It is not possible to start tasks without a framework. Marathon is a framework. Usually the framework process is not started as a Mesos task, but instead started by e.g., init.d/systemd, so the framework itself is not supervised by Mesos. Users can start Mesos tasks by e.g., hitting Marathon endpoints. Marathon then starts a task with Mesos and makes sure that the task keeps running (similar to how e.g., systemd might make sure that the Marathon framework process is running). Frameworks can talk to Mesos with two different APIs (often called v0 and v1); v0 is a C++ API while v1 communicates over HTTP. v1 is not a REST API. To launch Mesos tasks you will need a framework. You can either write a framework yourself, or use an existing one (there’s one list of some frameworks here, https://mesos.apache.org/documentation/latest/frameworks/). Cheers, Benjamin
