[jira] [Commented] (MESOS-7735) The master crashes when state endpoint is hit during a task authorization.

2017-07-05 Thread Michael Park (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074575#comment-16074575
 ] 

Michael Park commented on MESOS-7735:
-

{noformat}
commit e097f21124d16c80544af7aabc83bab5fb712453
Author: Michael Park 
Date:   Thu Jun 29 23:49:10 2017 -0700

Performed validation/normalization of `Resource`s before authorization.

Review: https://reviews.apache.org/r/60564
{noformat}
{noformat}
commit 63c8b1d63be363e96d7c2672d9d65b9d67ca48ed
Author: Michael Park 
Date:   Thu Jun 29 23:30:16 2017 -0700

Updated `validateAndNormalizeResources` to operate on `Operation`s.

Review: https://reviews.apache.org/r/60563
{noformat}
{noformat}
commit 710b72179938cac2100c90277ce7ced5c8ca3401
Author: Michael Park 
Date:   Thu Jun 29 20:53:59 2017 -0700

Updated `accept` to perform operation adjustment in one place.

It used to be that the minor adjustments that were made to operations
were done in various places across `accept` and `_accept`.

The "executor-injection" for LAUNCH_GROUP was at the beginning of
`accept`, "allocation-info-injection" for MULTI_ROLE was after offer
validation, and "health-check-injection" for LAUNCH was in `_accept`.

The `Master::accept` function is now broken down into distinct
"metrics accounting", "offer validation", "operation-adjustments", and
"authorization" stages.

Review: https://reviews.apache.org/r/60562
{noformat}

> The master crashes when state endpoint is hit during a task authorization.
> --
>
> Key: MESOS-7735
> URL: https://issues.apache.org/jira/browse/MESOS-7735
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Reporter: Michael Park
>Assignee: Michael Park
>Priority: Blocker
>
> With reservation refinement, the construction of {{Resources}} requires 
> {{Resource}} objects to have been validated and converted to the 
> "post-reservation-refinement" format. Generally speaking, validation and 
> conversion are the first steps we take with given {{Resource}} objects prior 
> to proceeding. In the master currently, we perform authorization first with 
> not-yet-validated, not-yet-converted {{Resource}} objects. During the 
> authorization phase, we add tasks with not-yet-validated, not-yet-converted 
> resources into {{framework->pendingTasks}} as well as 
> {{slave->pendingTasks}}. 
> (https://github.com/apache/mesos/blob/master/src/master/master.cpp#L3974-L3999).
>  If one hits the state endpoint on the master during this time, we get to 
> https://github.com/apache/mesos/blob/master/src/master/http.cpp#L278 which 
> tries to construct a {{Resources}} with {{taskInfo.resources()}} which is 
> not-yet-validated nor converted.
> I think the correct fix here is to perform validation / conversion prior to 
> authorization. The authorization code currently is written to carefully 
> inspect fields in both "pre-reservation-refinement" and 
> "post-reservation-refinement" formats. By performing validation / conversion 
> first, the authorization code would be simplified, and we're also much less 
> likely to make mistakes such as this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (MESOS-7735) The master crashes when state endpoint is hit during a task authorization.

2017-07-05 Thread Michael Park (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-7735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074576#comment-16074576
 ] 

Michael Park commented on MESOS-7735:
-

{noformat}
commit fcc15438c6a4a6885ba31845f33dc07b6f766a62
Author: Michael Park mp...@apache.org
Date:   Thu Jun 29 17:57:43 2017 -0700

Replaced a few raw `for` loops with `foreach`.

Review: https://reviews.apache.org/r/60561
{noformat}

> The master crashes when state endpoint is hit during a task authorization.
> --
>
> Key: MESOS-7735
> URL: https://issues.apache.org/jira/browse/MESOS-7735
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Reporter: Michael Park
>Assignee: Michael Park
>Priority: Blocker
>
> With reservation refinement, the construction of {{Resources}} requires 
> {{Resource}} objects to have been validated and converted to the 
> "post-reservation-refinement" format. Generally speaking, validation and 
> conversion are the first steps we take with given {{Resource}} objects prior 
> to proceeding. In the master currently, we perform authorization first with 
> not-yet-validated, not-yet-converted {{Resource}} objects. During the 
> authorization phase, we add tasks with not-yet-validated, not-yet-converted 
> resources into {{framework->pendingTasks}} as well as 
> {{slave->pendingTasks}}. 
> (https://github.com/apache/mesos/blob/master/src/master/master.cpp#L3974-L3999).
>  If one hits the state endpoint on the master during this time, we get to 
> https://github.com/apache/mesos/blob/master/src/master/http.cpp#L278 which 
> tries to construct a {{Resources}} with {{taskInfo.resources()}} which is 
> not-yet-validated nor converted.
> I think the correct fix here is to perform validation / conversion prior to 
> authorization. The authorization code currently is written to carefully 
> inspect fields in both "pre-reservation-refinement" and 
> "post-reservation-refinement" formats. By performing validation / conversion 
> first, the authorization code would be simplified, and we're also much less 
> likely to make mistakes such as this.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)