Repository: mesos Updated Branches: refs/heads/master a926cf3da -> cfeabec58
Added documentation for default executor and LAUNCH_GROUP event. This is very minimal documentation mainly around the new framework APIs and the default executor. Review: https://reviews.apache.org/r/52901 Project: http://git-wip-us.apache.org/repos/asf/mesos/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/cfeabec5 Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/cfeabec5 Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/cfeabec5 Branch: refs/heads/master Commit: cfeabec58fb2a87076f0a2cf4d46cdd02510bce4 Parents: eabb10c Author: Vinod Kone <vinodk...@gmail.com> Authored: Fri Oct 14 17:01:58 2016 -0700 Committer: Vinod Kone <vinodk...@gmail.com> Committed: Fri Oct 14 17:04:11 2016 -0700 ---------------------------------------------------------------------- CHANGELOG | 13 +++++++++ docs/app-framework-development-guide.md | 41 ++++++++++++++++++++++++++++ docs/executor-http-api.md | 38 ++++++++++++++++++++++++++ docs/scheduler-http-api.md | 4 +-- 4 files changed, 94 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/mesos/blob/cfeabec5/CHANGELOG ---------------------------------------------------------------------- diff --git a/CHANGELOG b/CHANGELOG index 1b8fa34..7ffd223 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -2,6 +2,15 @@ Release Notes - Mesos - Version 1.1.0 (WIP) -------------------------------------------- This release contains the following new features: + * [MESOS-2449] - **Experimental** support for launching a group of tasks + via a new `LAUNCH_GROUP` Offer operation. Mesos will guarantee that either + all tasks or none of the tasks in the group are delivered to the executor. + Executors receive the task group via a new `LAUNCH_GROUP` event. + + * [MESOS-6077] - **Experimental** A new default executor is introduced which + frameworks can use to launch task groups as nested containers. All the + nested containers share resources likes cpu, memory, network and volumes. + * [MESOS-2533] - **Experimental** support for HTTP and HTTPS health checks. Executors may now use the updated `HealthCheck` protobuf to implement HTTP(S) health checks. Both default executors (command and docker) leverage @@ -37,6 +46,10 @@ Additional API Changes: failures will now result in a `500 Internal Server Error` rather than a `503 Service Unavailable`. + * [MESOS-6241] - New API calls (LAUNCH_NESTED_CONTAINER, + KILL_NESTED_CONTAINER and WAIT_NESTED_CONTAINER) have been added to the + v1 Agent API to manage nested containers within an executor container. + Release Notes - Mesos - Version 1.0.2 -------------------------------------------- http://git-wip-us.apache.org/repos/asf/mesos/blob/cfeabec5/docs/app-framework-development-guide.md ---------------------------------------------------------------------- diff --git a/docs/app-framework-development-guide.md b/docs/app-framework-development-guide.md index c76f01a..de92c75 100644 --- a/docs/app-framework-development-guide.md +++ b/docs/app-framework-development-guide.md @@ -314,6 +314,47 @@ Note that the agent will derive an `ExecutorInfo` from the `TaskInfo` and additionally copy fields (e.g., `Labels`) from `TaskInfo` into the new `ExecutorInfo`. This `ExecutorInfo` is only visible on the agent. + +### Using the Mesos Default Executor + +Since Mesos 1.1, a new built-in default executor (**experimental**) is available that +can execute a group of tasks. Just like the command executor the tasks can be shell +commands or Docker containers. + +The current semantics of the default executor are as folows: + +-- Tasks are launched as nested containers underneath the executor container. + +-- Task containers and executor container share resources like cpu, memory, + network and volumes. + +-- There is no resource isolation between different tasks within an executor. + Tasks' resources are added to the executor container. + +-- If any of the tasks exits with a non-zero exit code, all the tasks in the task group + are killed and the executor shuts down. + +-- Multiple task groups are not supported. + + +Once the default executor is considered **stable**, the command executor will be deprecated in favor of it. + +Any scheduler can make use of the Mesos default executor by setting `ExecutorInfo.type` +to `DEFAULT` when launching a group of tasks using the `LAUNCH_GROUP` offer operation. +If `DEFAULT` executor is explicitly specified when using `LAUNCH` offer operation, command +executor is used instead of the default executor. This might change in the future when the default +executor gets support for handling `LAUNCH` operation. + + +~~~{.proto} +message ExecutorInfo { + ... + optional Type type = 15; + ... +} +~~~ + + ### Creating a custom Framework Executor If your framework has special requirements, you might want to provide your own http://git-wip-us.apache.org/repos/asf/mesos/blob/cfeabec5/docs/executor-http-api.md ---------------------------------------------------------------------- diff --git a/docs/executor-http-api.md b/docs/executor-http-api.md index 50b4cb4..a3b8d6f 100644 --- a/docs/executor-http-api.md +++ b/docs/executor-http-api.md @@ -260,6 +260,44 @@ LAUNCH Event (JSON) } ``` +### LAUNCH_GROUP + +This **experimental** event was added in 1.1.0. + +Sent by the agent whenever it needs to assign a new task group to the executor. The executor is required to send `UPDATE` messages back to the agent indicating the success or failure of each of the tasks in the group. + +The executor must maintain a list of unacknowledged tasks (see `LAUNCH` section above). + +``` +LAUNCH_GROUP Event (JSON) + +<event-length> +{ + "type": "LAUNCH_GROUP", + "launch_group": { + "task_group" : { + "tasks" : [ + "task": { + "name": "dummy-task", + "task_id": { + "value": "d40f3f3e-bbe3-44af-a230-4cb1eae72f67" + }, + "agent_id": { + "value": "f1c9cdc5-195e-41a7-a0d7-adaa9af07f81" + }, + "command": { + "value": "sleep", + "arguments": [ + "100" + ] + } + } + ] + } + } +} +``` + ### KILL The `KILL` event is sent whenever the scheduler needs to stop execution of a specific task. The executor is required to send a terminal update (e.g., `TASK_FINISHED`, `TASK_KILLED` or `TASK_FAILED`) back to the agent once it has stopped/killed the task. Mesos will mark the task resources as freed once the terminal update is received. http://git-wip-us.apache.org/repos/asf/mesos/blob/cfeabec5/docs/scheduler-http-api.md ---------------------------------------------------------------------- diff --git a/docs/scheduler-http-api.md b/docs/scheduler-http-api.md index a27965a..958cfc5 100644 --- a/docs/scheduler-http-api.md +++ b/docs/scheduler-http-api.md @@ -161,7 +161,7 @@ HTTP/1.1 202 Accepted ``` ### ACCEPT -Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, its resources are considered allocated to the framework. Also, any of the offer's resources not used in the `ACCEPT` call (e.g., to launch a task) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.). +Sent by the scheduler when it accepts offer(s) sent by the master. The `ACCEPT` request includes the type of operations (e.g., launch task, launch task group, reserve resources, create volumes) that the scheduler wants to perform on the offers. Note that until the scheduler replies (accepts or declines) to an offer, its resources are considered allocated to the framework. Also, any of the offer's resources not used in the `ACCEPT` call (e.g., to launch a task or task group) are considered declined and might be reoffered to other frameworks. In other words, the same `OfferID` cannot be used in more than one `ACCEPT` call. These semantics might change when we add new features to Mesos (e.g., persistence, reservations, optimistic offers, resizeTask, etc.). ``` ACCEPT Request (JSON): @@ -272,7 +272,7 @@ HTTP/1.1 202 Accepted ``` ### KILL -Sent by the scheduler to kill a specific task. If the scheduler has a custom executor, the kill is forwarded to the executor; it is up to the executor to kill the task and send a `TASK_KILLED` (or `TASK_FAILED`) update. Mesos releases the resources for a task once it receives a terminal update for the task. If the task is unknown to the master, a `TASK_LOST` will be generated. +Sent by the scheduler to kill a specific task. If the scheduler has a custom executor, the kill is forwarded to the executor; it is up to the executor to kill the task and send a `TASK_KILLED` (or `TASK_FAILED`) update. If the task hasn't yet been delivered to the executor when Mesos master or agent receives the kill request, a `TASK_KILLED` is generated and the task launch is not forwarded to the executor. Note that if the task belongs to a task group, killing of one task results in all tasks in the task group being killed. Mesos releases the resources for a task once it receives a terminal update for the task. If the task is unknown to the master, a `TASK_LOST` will be generated. ``` KILL Request (JSON):
