[
https://issues.apache.org/jira/browse/MESOS-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022937#comment-17022937
]
Benjamin Bannier edited comment on MESOS-10084 at 1/27/20 2:27 PM:
-------------------------------------------------------------------
{{1.5.x}}
{noformat}
commit 2f2146ac61abd54bc3296d8cbb5429cd10584db1
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 6294c319047f4c23ac28c9f20c39c88dfad2a66b
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit 095021dcadde9a19ed3298c65dcbed303651faa8
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.6.x}}:
{noformat}
commit 7e4d380c11c20f4b9e20b06f6b6c67a4657af24b
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 305f2b5e88ed9256b60a02afbdad06e2333937b7
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit efbb1e697af7082f3da1d509df9cd25ccbe0aab8
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.7.x}}:
{noformat}
commit 47f2a11e6ade71c455efe285898b52c256681e31
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 989ccb5c04c25dfde2d9c530cc84218268c403a6
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit a906d7d51b15193dd0742e1396dc209eb1bb09dc
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.8.x}}:
{noformat}
commit a66419bbab2cea8c7cf1b7663cd6f776f4095530
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 4fffb49c45e550c149e405574eee3a5870106bd4
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit 35451b65c4f77c17e8338c6deb1245204b944242
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.9.x}}:
{noformat}
commit 298d1d70e5a4aa383650ce3c81f9db64aaded2d4
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 631de3d4df9bd8e24edc6687c544d5bb45be0198
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit 603525654d2aea5ed46c47a9da2ce2dce07188ab
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
was (Author: bbannier):
{{1.6.x}}:
{noformat}
commit 7e4d380c11c20f4b9e20b06f6b6c67a4657af24b
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 305f2b5e88ed9256b60a02afbdad06e2333937b7
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit efbb1e697af7082f3da1d509df9cd25ccbe0aab8
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.7.x}}:
{noformat}
commit 47f2a11e6ade71c455efe285898b52c256681e31
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 989ccb5c04c25dfde2d9c530cc84218268c403a6
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit a906d7d51b15193dd0742e1396dc209eb1bb09dc
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.8.x}}:
{noformat}
commit a66419bbab2cea8c7cf1b7663cd6f776f4095530
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 4fffb49c45e550c149e405574eee3a5870106bd4
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit 35451b65c4f77c17e8338c6deb1245204b944242
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
{{1.9.x}}:
{noformat}
commit 298d1d70e5a4aa383650ce3c81f9db64aaded2d4
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:57 2020 +0100
Remembered whether an executor was agent-generated.
This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.
Review: https://reviews.apache.org/r/72035/
commit 631de3d4df9bd8e24edc6687c544d5bb45be0198
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:51 2020 +0100
Sync'd whether an executor was generated to and from disk.
This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.
Review: https://reviews.apache.org/r/72034/
commit 603525654d2aea5ed46c47a9da2ce2dce07188ab
Author: Benjamin Bannier <[email protected]>
Date: Thu Jan 23 14:19:43 2020 +0100
Decoupled detection of generated executors from Mesos install location.
We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`. This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.
This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.
Review: https://reviews.apache.org/r/72033/
{noformat}
> Detecting whether executor is generated for command task should work when the
> launcher_dir changes
> --------------------------------------------------------------------------------------------------
>
> Key: MESOS-10084
> URL: https://issues.apache.org/jira/browse/MESOS-10084
> Project: Mesos
> Issue Type: Bug
> Reporter: Andrei Sekretenko
> Assignee: Benjamin Bannier
> Priority: Critical
> Fix For: 1.6.3, 1.7.3, 1.8.2, 1.9.1, 1.10.0
>
>
> As currently implemented, on recovery Mesos agent determines that the
> executor is generated for command task by comparing the executor command with
> a current path to Mesos executor:
> https://github.com/apache/mesos/blob/1.7.x/src/slave/slave.cpp#L9635
> During upgrade of production cluster we observed this check to break due to
> the new launcher_dir being different from the one of checkpointed executor.
> This can cause problems of various kind: for example, after such upgrade,
> Mesos master can begin to treat the checkpointed command executors as subject
> to resource quota.
> Design considerations:
> - proper solution is to checkpoint the flag indicating whether the executor
> is a command/docker one.
> - for correct upgrade from older Mesos versions, we will need some kind of
> workaround to detect command executors after upgrade; the workaround logic
> should be skipped if there is a checkpointed flag.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
