[jira] [Assigned] (MESOS-1733) Change the stout path utility to declare a single, variadic 'join' function instead of several separate declarations of various discrete arities
[ https://issues.apache.org/jira/browse/MESOS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-1733: - Assignee: Anand Mazumdar (was: Cody Maloney) > Change the stout path utility to declare a single, variadic 'join' function > instead of several separate declarations of various discrete arities > > > Key: MESOS-1733 > URL: https://issues.apache.org/jira/browse/MESOS-1733 > Project: Mesos > Issue Type: Improvement > Components: build, stout >Reporter: Patrick Reilly >Assignee: Anand Mazumdar > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1733) Change the stout path utility to declare a single, variadic 'join' function instead of several separate declarations of various discrete arities
[ https://issues.apache.org/jira/browse/MESOS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-1733: -- Shepherd: Adam B (was: Benjamin Hindman) > Change the stout path utility to declare a single, variadic 'join' function > instead of several separate declarations of various discrete arities > > > Key: MESOS-1733 > URL: https://issues.apache.org/jira/browse/MESOS-1733 > Project: Mesos > Issue Type: Improvement > Components: build, stout >Reporter: Patrick Reilly >Assignee: Cody Maloney > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1988) Scheduler driver should not generate TASK_LOST when disconnected from master
[ https://issues.apache.org/jira/browse/MESOS-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588666#comment-14588666 ] Anand Mazumdar commented on MESOS-1988: --- It seems that the library (src/scheduler/scheduler.cpp) already does the right thing by dropping calls silently. I would go ahead and nuke the second overload(...) that took a TaskInfo as a argument as its no longer being used in the code. > Scheduler driver should not generate TASK_LOST when disconnected from master > > > Key: MESOS-1988 > URL: https://issues.apache.org/jira/browse/MESOS-1988 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: twitter > > Currently, the driver replies to launchTasks() with TASK_LOST if it detects > that it is disconnected from the master. After MESOS-1972 lands, this will be > the only place where driver generates TASK_LOST. See MESOS-1972 for more > context. > This fix is targeted for 0.22.0 to give frameworks time to implement > reconciliation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1988) Scheduler driver should not generate TASK_LOST when disconnected from master
[ https://issues.apache.org/jira/browse/MESOS-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588993#comment-14588993 ] Anand Mazumdar commented on MESOS-1988: --- Deleted OverLoad for review here : https://reviews.apache.org/r/35538 Left: - Send email to dev mailing list to apprise them of the change in driver in (0.24?) - Delete the relevant fragment of code that returns TASK_LOST from sched/sched.cpp. > Scheduler driver should not generate TASK_LOST when disconnected from master > > > Key: MESOS-1988 > URL: https://issues.apache.org/jira/browse/MESOS-1988 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: twitter > > Currently, the driver replies to launchTasks() with TASK_LOST if it detects > that it is disconnected from the master. After MESOS-1972 lands, this will be > the only place where driver generates TASK_LOST. See MESOS-1972 for more > context. > This fix is targeted for 0.22.0 to give frameworks time to implement > reconciliation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2295) Implement the Call endpoint on Slave
[ https://issues.apache.org/jira/browse/MESOS-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14592131#comment-14592131 ] Anand Mazumdar commented on MESOS-2295: --- Related Shelved review from Alexander : https://reviews.apache.org/r/33824 > Implement the Call endpoint on Slave > > > Key: MESOS-2295 > URL: https://issues.apache.org/jira/browse/MESOS-2295 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: haosdent > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2295) Implement the Call endpoint on Slave
[ https://issues.apache.org/jira/browse/MESOS-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2295: - Assignee: Anand Mazumdar (was: haosdent) > Implement the Call endpoint on Slave > > > Key: MESOS-2295 > URL: https://issues.apache.org/jira/browse/MESOS-2295 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2296) Implement the Events endpoint on slave
[ https://issues.apache.org/jira/browse/MESOS-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2296: - Assignee: Anand Mazumdar (was: haosdent) > Implement the Events endpoint on slave > -- > > Key: MESOS-2296 > URL: https://issues.apache.org/jira/browse/MESOS-2296 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2906) Slave : Synchronous Validation for Calls
Anand Mazumdar created MESOS-2906: - Summary: Slave : Synchronous Validation for Calls Key: MESOS-2906 URL: https://issues.apache.org/jira/browse/MESOS-2906 Project: Mesos Issue Type: Task Reporter: Anand Mazumdar Assignee: Anand Mazumdar /call endpoint on the slave will return a 202 accepted code but has to do some basic validations before. In case of invalidation it will return a 4xx code. - We need to create the required infrastructure to validate the request and then process it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
Anand Mazumdar created MESOS-2907: - Summary: Slave : Create Basic Functionality to handle /call endpoint Key: MESOS-2907 URL: https://issues.apache.org/jira/browse/MESOS-2907 Project: Mesos Issue Type: Task Reporter: Anand Mazumdar Assignee: Anand Mazumdar This is the first basic step in ensuring the basic /call functionality: processing a POST /call and returning: 202 if all goes well; 401 if not authorized; and 403 if the request is malformed. Also , we might need to store some identifier which enables us to reject calls to /call if the executor has not issues a SUBSCRIBE/RESUBSCRIBE Request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2295) Implement the Call endpoint on Slave
[ https://issues.apache.org/jira/browse/MESOS-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596398#comment-14596398 ] Anand Mazumdar commented on MESOS-2295: --- [~marco-mesos] Just created 2 smaller tasks ( mirror image ) of the one's for master endpoint. > Implement the Call endpoint on Slave > > > Key: MESOS-2295 > URL: https://issues.apache.org/jira/browse/MESOS-2295 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2907: -- Description: This is the first basic step in ensuring the basic /call functionality: processing a POST /call and returning: 202 if all goes well; 401 if not authorized; and 403 if the request is malformed. Also , we might need to store some identifier which enables us to reject calls to /call if the client has not issued a SUBSCRIBE/RESUBSCRIBE Request. was: This is the first basic step in ensuring the basic /call functionality: processing a POST /call and returning: 202 if all goes well; 401 if not authorized; and 403 if the request is malformed. Also , we might need to store some identifier which enables us to reject calls to /call if the executor has not issues a SUBSCRIBE/RESUBSCRIBE Request. > Slave : Create Basic Functionality to handle /call endpoint > --- > > Key: MESOS-2907 > URL: https://issues.apache.org/jira/browse/MESOS-2907 > Project: Mesos > Issue Type: Task >Reporter: Anand Mazumdar >Assignee: Anand Mazumdar > Labels: HTTP, mesosphere > > This is the first basic step in ensuring the basic /call functionality: > processing a > POST /call > and returning: > 202 if all goes well; > 401 if not authorized; and > 403 if the request is malformed. > Also , we might need to store some identifier which enables us to reject > calls to /call if the client has not issued a SUBSCRIBE/RESUBSCRIBE Request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596416#comment-14596416 ] Anand Mazumdar commented on MESOS-2907: --- This ticket only tracks the work done for the slave. The one for the master is here : https://issues.apache.org/jira/browse/MESOS-2860 . Most of the work, however, should be applicable to both. > Slave : Create Basic Functionality to handle /call endpoint > --- > > Key: MESOS-2907 > URL: https://issues.apache.org/jira/browse/MESOS-2907 > Project: Mesos > Issue Type: Task >Reporter: Anand Mazumdar >Assignee: Anand Mazumdar > Labels: HTTP, mesosphere > > This is the first basic step in ensuring the basic /call functionality: > processing a > POST /call > and returning: > 202 if all goes well; > 401 if not authorized; and > 403 if the request is malformed. > Also , we might need to store some identifier which enables us to reject > calls to /call if the client has not issued a SUBSCRIBE/RESUBSCRIBE Request. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596511#comment-14596511 ] Anand Mazumdar commented on MESOS-2708: --- I would take it up. Waiting on [~arojas] to give me "editing" permissions on the document. > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Alexander Rojas > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2296) Implement the Events stream on slave for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2296: -- Summary: Implement the Events stream on slave for Call endpoint (was: Implement the Events endpoint on slave) > Implement the Events stream on slave for Call endpoint > -- > > Key: MESOS-2296 > URL: https://issues.apache.org/jira/browse/MESOS-2296 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2296) Implement the Events stream on slave for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2296: -- Assignee: (was: Anand Mazumdar) > Implement the Events stream on slave for Call endpoint > -- > > Key: MESOS-2296 > URL: https://issues.apache.org/jira/browse/MESOS-2296 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2294: - Assignee: Anand Mazumdar > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1988) Scheduler driver should not generate TASK_LOST when disconnected from master
[ https://issues.apache.org/jira/browse/MESOS-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-1988: -- Fix Version/s: 0.24.0 > Scheduler driver should not generate TASK_LOST when disconnected from master > > > Key: MESOS-1988 > URL: https://issues.apache.org/jira/browse/MESOS-1988 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere, twitter > Fix For: 0.24.0 > > > Currently, the driver replies to launchTasks() with TASK_LOST if it detects > that it is disconnected from the master. After MESOS-1972 lands, this will be > the only place where driver generates TASK_LOST. See MESOS-1972 for more > context. > This fix is targeted for 0.22.0 to give frameworks time to implement > reconciliation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606163#comment-14606163 ] Anand Mazumdar commented on MESOS-2552: --- This was needed for testing the call HTTP endpoint on master. Submitted a diff up for review : https://reviews.apache.org/r/36013 ( is still a work in progress ) > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2552: - Assignee: Anand Mazumdar > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2552: -- Sprint: Mesosphere Sprint 13 Labels: mesosphere (was: ) > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2294: -- Labels: mesosphere twitter (was: twitter) > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere, twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615371#comment-14615371 ] Anand Mazumdar commented on MESOS-2294: --- Working to get the basic loop , client->subscribe->subscribed(response) back on the event stream and would send in a patch for that. > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere, twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2294: -- Sprint: Mesosphere Sprint 14 > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere, twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619280#comment-14619280 ] Anand Mazumdar commented on MESOS-2294: --- Initial review for just getting back a subscribed event on the stream : https://reviews.apache.org/r/36318/ > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere, twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2294: -- Shepherd: Benjamin Mahler > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2552: -- Shepherd: Benjamin Mahler > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3067) Implement a streaming response decoder for events stream
Anand Mazumdar created MESOS-3067:
-
Summary: Implement a streaming response decoder for events stream
Key: MESOS-3067
URL: https://issues.apache.org/jira/browse/MESOS-3067
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Assignee: Benjamin Mahler
We need a streaming response decoder to de-serialize chunks sent from the
master on the events stream.
>From the HTTP API design doc:
Master encodes each Event in RecordIO format, i.e. a string representation of
length of the event in bytes followed by JSON or binary Protobuf (possibly
compressed) encoded event.
As of now for getting the basic features right , this is being done in the
test-cases:
{code}
auto reader = response.get().reader;
ASSERT_SOME(reader);
Future eventFuture = reader.get().read();
AWAIT_READY(eventFuture);
Event event;
event.ParseFromString(eventFuture.get());
{code}
Two things need to happen:
- We need master to emit events in RecordIO format i.e. event size followed by
the serialized event instead of just the serialized events as is the case now.
- The decoder class should then abstract away the logic of reading the response
and de-serializing events from the stream.
Ideally, the decoder should work with both "json" and "protobuf" responses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2708: - Assignee: Anand Mazumdar (was: Alexander Rojas) > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Anand Mazumdar > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633869#comment-14633869 ] Anand Mazumdar commented on MESOS-2708: --- My bad, I had missed this comment completely. I would take this up when we start to focus on the JIRA items for Executor HTTP API again. > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Alexander Rojas > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2911) Add an Event message handler to scheduler library
[ https://issues.apache.org/jira/browse/MESOS-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2911: - Assignee: Anand Mazumdar (was: Benjamin Mahler) > Add an Event message handler to scheduler library > - > > Key: MESOS-2911 > URL: https://issues.apache.org/jira/browse/MESOS-2911 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > > Adding this handler lets master send Event messages to the library. > See MESOS-2909 for additional context. > This ticket only tracks the installation of the handler and maybe handling of > a single event for testing. Additional events handling will be captured in a > different ticket(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634057#comment-14634057 ] Anand Mazumdar commented on MESOS-2294: --- The part in review is a very small part of the JIRA i.e. just handles subscribe->subscribed calls. Would send in more patches for other events once we agree on the design/code semantics of the review I sent out. > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634061#comment-14634061 ] Anand Mazumdar commented on MESOS-2552: --- I am pausing the progress on this one. It would be a better idea to resume work on this one once the master can understand calls/send events on the event stream. > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2552: -- Sprint: (was: Mesosphere Sprint 15) > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3119) Remove pthread specific code from Libprocess
[ https://issues.apache.org/jira/browse/MESOS-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639537#comment-14639537 ] Anand Mazumdar commented on MESOS-3119: --- Can we add some context/background here ? It's very hard to reason about the motivation of the changes from reviews linking to these. > Remove pthread specific code from Libprocess > > > Key: MESOS-3119 > URL: https://issues.apache.org/jira/browse/MESOS-3119 > Project: Mesos > Issue Type: Improvement > Components: libprocess >Reporter: Joris Van Remoortere >Assignee: Joris Van Remoortere > Labels: libprocess, mesosphere, windows > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3169) FrameworkInfo should only be updated if the re-registration is valid
[
https://issues.apache.org/jira/browse/MESOS-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646489#comment-14646489
]
Anand Mazumdar commented on MESOS-3169:
---
The `FrameworkErrorMessage` generated from `failoverFramework` is sent to the
old scheduler. So , we should go ahead with updating the framework info
correctly for that case as we are doing now.
The only point of contention hence is this:
{code }else if (from != framework->pid); {code}
I guess the easiest fix would be to update the info at the end of the function
rather then at the beginning unless I am missing something ?
> FrameworkInfo should only be updated if the re-registration is valid
>
>
> Key: MESOS-3169
> URL: https://issues.apache.org/jira/browse/MESOS-3169
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 0.23.0
>Reporter: Joris Van Remoortere
> Labels: framework, master, mesosphere
>
> See Ben Mahler's comment in https://reviews.apache.org/r/32961/
> FrameworkInfo should not be updated if the re-registration is invalid. This
> can happen in a few cases under the branching logic, so this requires some
> refactoring.
> Notice that a {code}FrameworkErrorMessage{code} can be generated both inside
> {code}else if (from != framework->pid){code} as well as from inside
> {code}failoverFramework(framework, from);{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3169) FrameworkInfo should only be updated if the re-registration is valid
[
https://issues.apache.org/jira/browse/MESOS-3169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646489#comment-14646489
]
Anand Mazumdar edited comment on MESOS-3169 at 7/29/15 5:48 PM:
The `FrameworkErrorMessage` generated from `failoverFramework` is sent to the
old scheduler. So , we should go ahead with updating the framework info
correctly for that case as we are doing now.
The only point of contention hence is this:
{code}else if (from != framework->pid); {code}
I guess the easiest fix would be to update the info at the end of the function
rather then at the beginning unless I am missing something ?
was (Author: anandmazumdar):
The `FrameworkErrorMessage` generated from `failoverFramework` is sent to the
old scheduler. So , we should go ahead with updating the framework info
correctly for that case as we are doing now.
The only point of contention hence is this:
{code }else if (from != framework->pid); {code}
I guess the easiest fix would be to update the info at the end of the function
rather then at the beginning unless I am missing something ?
> FrameworkInfo should only be updated if the re-registration is valid
>
>
> Key: MESOS-3169
> URL: https://issues.apache.org/jira/browse/MESOS-3169
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 0.23.0
>Reporter: Joris Van Remoortere
> Labels: framework, master, mesosphere
>
> See Ben Mahler's comment in https://reviews.apache.org/r/32961/
> FrameworkInfo should not be updated if the re-registration is invalid. This
> can happen in a few cases under the branching logic, so this requires some
> refactoring.
> Notice that a {code}FrameworkErrorMessage{code} can be generated both inside
> {code}else if (from != framework->pid){code} as well as from inside
> {code}failoverFramework(framework, from);{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651324#comment-14651324
]
Anand Mazumdar commented on MESOS-2294:
---
{code}
commit 90b107a249169c6fc8b8d398b675ab9bd2df633b
Author: Anand Mazumdar
Date: Tue Jul 28 11:53:45 2015 -0700
Updated Framework struct in master for the http api.
This change refactors the Framework struct in master to introduce
support for http frameworks:
* 'pid' becomes a optional field.
* Added optional 'http' field.
Review: https://reviews.apache.org/r/36318
{code}
> Implement the Events stream on master for Call endpoint
> ---
>
> Key: MESOS-2294
> URL: https://issues.apache.org/jira/browse/MESOS-2294
> Project: Mesos
> Issue Type: Task
>Reporter: Vinod Kone
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651326#comment-14651326 ] Anand Mazumdar commented on MESOS-2294: --- Patch for subscribe->subscribed workflow : https://reviews.apache.org/r/36720 Working on re-registration equivalent for the subscribe call now. > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654020#comment-14654020 ] Anand Mazumdar commented on MESOS-2294: --- Tests for subscribe/failover workflow: https://reviews.apache.org/r/37082 > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2860) Create the basic infrastructure to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654619#comment-14654619
]
Anand Mazumdar commented on MESOS-2860:
---
{code}
commit 8e4d1c6e4fd1be2fe05db045f034b84bf19e04af
Author: Anand Mazumdar
Date: Tue Aug 4 13:16:58 2015 -0700
Added /call parsing and validation.
Review: https://reviews.apache.org/r/36720
{code}
> Create the basic infrastructure to handle /call endpoint
>
>
> Key: MESOS-2860
> URL: https://issues.apache.org/jira/browse/MESOS-2860
> Project: Mesos
> Issue Type: Story
> Components: master
>Reporter: Marco Massenzio
>Assignee: Isabel Jimenez
> Labels: mesosphere
>
> This is the first basic step in ensuring the basic {{/call}} functionality:
> processing a
> {noformat}
> POST /call
> {noformat}
> and returning:
> - {{202}} if all goes well;
> - {{401}} if not authorized; and
> - {{403}} if the request is malformed.
> We'll get more sophisticated as the work progressed (eg, supporting {{415}}
> if the content-type is not of the right kind).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2552: -- Sprint: Mesosphere Sprint 16 > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14682122#comment-14682122 ] Anand Mazumdar commented on MESOS-2552: --- Review Chain: https://reviews.apache.org/r/37298 https://reviews.apache.org/r/37300 https://reviews.apache.org/r/37301 Pending: https://reviews.apache.org/r/37302 https://reviews.apache.org/r/37303 https://reviews.apache.org/r/37304 https://reviews.apache.org/r/37328/ https://reviews.apache.org/r/37335/ > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (MESOS-2552) C++ Scheduler library should send HTTP Calls to master
[ https://issues.apache.org/jira/browse/MESOS-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2552: -- Comment: was deleted (was: This was needed for testing the call HTTP endpoint on master. Submitted a diff up for review : https://reviews.apache.org/r/36013 ( is still a work in progress )) > C++ Scheduler library should send HTTP Calls to master > -- > > Key: MESOS-2552 > URL: https://issues.apache.org/jira/browse/MESOS-2552 > Project: Mesos > Issue Type: Bug >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > > Once the scheduler library sends Call messages, we should update it to send > Calls as HTTP requests to "/call" endpoint on master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3263) SchedulerTask.KillTest fails for JSON Requests
Anand Mazumdar created MESOS-3263: - Summary: SchedulerTask.KillTest fails for JSON Requests Key: MESOS-3263 URL: https://issues.apache.org/jira/browse/MESOS-3263 Project: Mesos Issue Type: Bug Reporter: Anand Mazumdar Fix For: 0.24.0 Currently, SchedulerTests.KillTask fails when the ContentType specified is JSON in the request. The crash happens in Master when it tries to process the Acknowledge call from client. The sent UUID escaped string in JSON from the client is unable to be correctly parsed by the master leading to the crash. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3311) SlaveTest.HTTPSchedulerSlaveRestart
[
https://issues.apache.org/jira/browse/MESOS-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar reassigned MESOS-3311:
-
Assignee: Anand Mazumdar
> SlaveTest.HTTPSchedulerSlaveRestart
> ---
>
> Key: MESOS-3311
> URL: https://issues.apache.org/jira/browse/MESOS-3311
> Project: Mesos
> Issue Type: Bug
> Components: test
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/729/consoleFull
>Reporter: Vinod Kone
>Assignee: Anand Mazumdar
> Labels: flaky-test
>
> Observed on ASF CI
> {code}
> [ RUN ] SlaveTest.HTTPSchedulerSlaveRestart
> Using temporary directory '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA'
> I0825 22:07:36.809872 27610 leveldb.cpp:176] Opened db in 3.751801ms
> I0825 22:07:36.85 27610 leveldb.cpp:183] Compacted db in 1.2194ms
> I0825 22:07:36.811175 27610 leveldb.cpp:198] Created db iterator in 30669ns
> I0825 22:07:36.811197 27610 leveldb.cpp:204] Seeked to beginning of db in
> 7829ns
> I0825 22:07:36.811208 27610 leveldb.cpp:273] Iterated through 0 keys in the
> db in 6017ns
> I0825 22:07:36.811245 27610 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0825 22:07:36.811722 27638 recover.cpp:449] Starting replica recovery
> I0825 22:07:36.811980 27638 recover.cpp:475] Replica is in EMPTY status
> I0825 22:07:36.813033 27641 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I0825 22:07:36.813355 27635 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I0825 22:07:36.813756 27628 recover.cpp:566] Updating replica status to
> STARTING
> I0825 22:07:36.814434 27636 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 570160ns
> I0825 22:07:36.814471 27636 replica.cpp:323] Persisted replica status to
> STARTING
> I0825 22:07:36.814743 27642 recover.cpp:475] Replica is in STARTING status
> I0825 22:07:36.814965 27638 master.cpp:378] Master
> 20150825-220736-234885548-51219-27610 (09c6504e3a31) started on
> 172.17.0.14:51219
> I0825 22:07:36.814999 27638 master.cpp:380] Flags at startup: --acls=""
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="25secs"
> --registry_strict="true" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/mesos/mesos-0.25.0/_inst/share/mesos/webui"
> --work_dir="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/master"
> --zk_session_timeout="10secs"
> I0825 22:07:36.815347 27638 master.cpp:425] Master only allowing
> authenticated frameworks to register
> I0825 22:07:36.815371 27638 master.cpp:430] Master only allowing
> authenticated slaves to register
> I0825 22:07:36.815402 27638 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials'
> I0825 22:07:36.815634 27632 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I0825 22:07:36.815752 27638 master.cpp:469] Using default 'crammd5'
> authenticator
> I0825 22:07:36.815904 27638 master.cpp:506] Authorization enabled
> I0825 22:07:36.815979 27643 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I0825 22:07:36.816185 27637 whitelist_watcher.cpp:79] No whitelist given
> I0825 22:07:36.816186 27641 hierarchical.hpp:346] Initialized hierarchical
> allocator process
> I0825 22:07:36.816519 27630 recover.cpp:566] Updating replica status to VOTING
> I0825 22:07:36.817258 27639 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 475231ns
> I0825 22:07:36.817296 27639 replica.cpp:323] Persisted replica status to
> VOTING
> I0825 22:07:36.817420 27637 master.cpp:1525] The newly elected leader is
> master@172.17.0.14:51219 with id 20150825-220736-234885548-51219-27610
> I0825 22:07:36.817467 27637 master.cpp:1538] Elected as the leading master!
> I0825 22:07:36.817483 27637 master.cpp:1308] Recovering from registrar
> I0825 22:07:36.817509 27635 recover.cpp:580] Successfully joined the Paxos
> group
> I0825 22:07:36.817708 27633 registrar
[jira] [Commented] (MESOS-3311) SlaveTest.HTTPSchedulerSlaveRestart
[
https://issues.apache.org/jira/browse/MESOS-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712113#comment-14712113
]
Anand Mazumdar commented on MESOS-3311:
---
Taking a look, looks related to the re-routing of ExecutorToFrameworkMessages
through master that we introduced for HTTP Frameworks. ( ac70a59, 9172a5f)
> SlaveTest.HTTPSchedulerSlaveRestart
> ---
>
> Key: MESOS-3311
> URL: https://issues.apache.org/jira/browse/MESOS-3311
> Project: Mesos
> Issue Type: Bug
> Components: test
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/729/consoleFull
>Reporter: Vinod Kone
>Assignee: Anand Mazumdar
> Labels: flaky-test
>
> Observed on ASF CI
> {code}
> [ RUN ] SlaveTest.HTTPSchedulerSlaveRestart
> Using temporary directory '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA'
> I0825 22:07:36.809872 27610 leveldb.cpp:176] Opened db in 3.751801ms
> I0825 22:07:36.85 27610 leveldb.cpp:183] Compacted db in 1.2194ms
> I0825 22:07:36.811175 27610 leveldb.cpp:198] Created db iterator in 30669ns
> I0825 22:07:36.811197 27610 leveldb.cpp:204] Seeked to beginning of db in
> 7829ns
> I0825 22:07:36.811208 27610 leveldb.cpp:273] Iterated through 0 keys in the
> db in 6017ns
> I0825 22:07:36.811245 27610 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0825 22:07:36.811722 27638 recover.cpp:449] Starting replica recovery
> I0825 22:07:36.811980 27638 recover.cpp:475] Replica is in EMPTY status
> I0825 22:07:36.813033 27641 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I0825 22:07:36.813355 27635 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I0825 22:07:36.813756 27628 recover.cpp:566] Updating replica status to
> STARTING
> I0825 22:07:36.814434 27636 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 570160ns
> I0825 22:07:36.814471 27636 replica.cpp:323] Persisted replica status to
> STARTING
> I0825 22:07:36.814743 27642 recover.cpp:475] Replica is in STARTING status
> I0825 22:07:36.814965 27638 master.cpp:378] Master
> 20150825-220736-234885548-51219-27610 (09c6504e3a31) started on
> 172.17.0.14:51219
> I0825 22:07:36.814999 27638 master.cpp:380] Flags at startup: --acls=""
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="25secs"
> --registry_strict="true" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/mesos/mesos-0.25.0/_inst/share/mesos/webui"
> --work_dir="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/master"
> --zk_session_timeout="10secs"
> I0825 22:07:36.815347 27638 master.cpp:425] Master only allowing
> authenticated frameworks to register
> I0825 22:07:36.815371 27638 master.cpp:430] Master only allowing
> authenticated slaves to register
> I0825 22:07:36.815402 27638 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials'
> I0825 22:07:36.815634 27632 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I0825 22:07:36.815752 27638 master.cpp:469] Using default 'crammd5'
> authenticator
> I0825 22:07:36.815904 27638 master.cpp:506] Authorization enabled
> I0825 22:07:36.815979 27643 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I0825 22:07:36.816185 27637 whitelist_watcher.cpp:79] No whitelist given
> I0825 22:07:36.816186 27641 hierarchical.hpp:346] Initialized hierarchical
> allocator process
> I0825 22:07:36.816519 27630 recover.cpp:566] Updating replica status to VOTING
> I0825 22:07:36.817258 27639 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 475231ns
> I0825 22:07:36.817296 27639 replica.cpp:323] Persisted replica status to
> VOTING
> I0825 22:07:36.817420 27637 master.cpp:1525] The newly elected leader is
> master@172.17.0.14:51219 with id 20150825-220736-234885548-51219-27610
> I0825 22:07:36.817467 27637 master.cpp:1538] Elected as the leading master!
> I0825 22:07:36.8174
[jira] [Commented] (MESOS-3311) SlaveTest.HTTPSchedulerSlaveRestart
[
https://issues.apache.org/jira/browse/MESOS-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712292#comment-14712292
]
Anand Mazumdar commented on MESOS-3311:
---
>From the logs, the slave sends in a "retried" re-registration request that
>triggers a FrameworkUpdateMessage again thereby re-writing the pid from
>0.0.0.0 back to the original scheduler pid.
Just pausing the clock to disable retries should be able to fix this.
> SlaveTest.HTTPSchedulerSlaveRestart
> ---
>
> Key: MESOS-3311
> URL: https://issues.apache.org/jira/browse/MESOS-3311
> Project: Mesos
> Issue Type: Bug
> Components: test
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/729/consoleFull
>Reporter: Vinod Kone
>Assignee: Anand Mazumdar
> Labels: flaky-test
>
> Observed on ASF CI
> {code}
> [ RUN ] SlaveTest.HTTPSchedulerSlaveRestart
> Using temporary directory '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA'
> I0825 22:07:36.809872 27610 leveldb.cpp:176] Opened db in 3.751801ms
> I0825 22:07:36.85 27610 leveldb.cpp:183] Compacted db in 1.2194ms
> I0825 22:07:36.811175 27610 leveldb.cpp:198] Created db iterator in 30669ns
> I0825 22:07:36.811197 27610 leveldb.cpp:204] Seeked to beginning of db in
> 7829ns
> I0825 22:07:36.811208 27610 leveldb.cpp:273] Iterated through 0 keys in the
> db in 6017ns
> I0825 22:07:36.811245 27610 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0825 22:07:36.811722 27638 recover.cpp:449] Starting replica recovery
> I0825 22:07:36.811980 27638 recover.cpp:475] Replica is in EMPTY status
> I0825 22:07:36.813033 27641 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I0825 22:07:36.813355 27635 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I0825 22:07:36.813756 27628 recover.cpp:566] Updating replica status to
> STARTING
> I0825 22:07:36.814434 27636 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 570160ns
> I0825 22:07:36.814471 27636 replica.cpp:323] Persisted replica status to
> STARTING
> I0825 22:07:36.814743 27642 recover.cpp:475] Replica is in STARTING status
> I0825 22:07:36.814965 27638 master.cpp:378] Master
> 20150825-220736-234885548-51219-27610 (09c6504e3a31) started on
> 172.17.0.14:51219
> I0825 22:07:36.814999 27638 master.cpp:380] Flags at startup: --acls=""
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="25secs"
> --registry_strict="true" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/mesos/mesos-0.25.0/_inst/share/mesos/webui"
> --work_dir="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/master"
> --zk_session_timeout="10secs"
> I0825 22:07:36.815347 27638 master.cpp:425] Master only allowing
> authenticated frameworks to register
> I0825 22:07:36.815371 27638 master.cpp:430] Master only allowing
> authenticated slaves to register
> I0825 22:07:36.815402 27638 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials'
> I0825 22:07:36.815634 27632 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I0825 22:07:36.815752 27638 master.cpp:469] Using default 'crammd5'
> authenticator
> I0825 22:07:36.815904 27638 master.cpp:506] Authorization enabled
> I0825 22:07:36.815979 27643 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I0825 22:07:36.816185 27637 whitelist_watcher.cpp:79] No whitelist given
> I0825 22:07:36.816186 27641 hierarchical.hpp:346] Initialized hierarchical
> allocator process
> I0825 22:07:36.816519 27630 recover.cpp:566] Updating replica status to VOTING
> I0825 22:07:36.817258 27639 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 475231ns
> I0825 22:07:36.817296 27639 replica.cpp:323] Persisted replica status to
> VOTING
> I0825 22:07:36.817420 27637 master.cpp:1525] The newly elected leader is
> master@172.17.0.14:51219 with id 20150825-220736-234885548
[jira] [Updated] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2708: -- Sprint: Mesosphere Sprint 17 > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Anand Mazumdar > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3332) Support HTTP Pipelining in libprocess (http::post)
Anand Mazumdar created MESOS-3332:
-
Summary: Support HTTP Pipelining in libprocess (http::post)
Key: MESOS-3332
URL: https://issues.apache.org/jira/browse/MESOS-3332
Project: Mesos
Issue Type: Task
Components: libprocess
Reporter: Anand Mazumdar
Currently , {{ http::post }} in libprocess, does not support HTTP pipelining.
Each call as of know sends in the {{ Connection: close }} header, thereby,
signaling to the server to close the TCP socket after the response.
We either need to create a new interface for supporting HTTP pipelining , or
modify the existing {{http::post}} to do so.
This is needed for the Scheduler/Executor library implementations to make sure
"Calls" are sent in order to the master. Currently, in order to do so, we send
in the next request only after we have received a response for an earlier call
that results in degraded performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3332) Support HTTP Pipelining in libprocess (http::post)
[
https://issues.apache.org/jira/browse/MESOS-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3332:
--
Description:
Currently , {{http::post}} in libprocess, does not support HTTP pipelining.
Each call as of know sends in the {{Connection: close}} header, thereby,
signaling to the server to close the TCP socket after the response.
We either need to create a new interface for supporting HTTP pipelining , or
modify the existing {{http::post}} to do so.
This is needed for the Scheduler/Executor library implementations to make sure
"Calls" are sent in order to the master. Currently, in order to do so, we send
in the next request only after we have received a response for an earlier call
that results in degraded performance.
was:
Currently , {{ http::post }} in libprocess, does not support HTTP pipelining.
Each call as of know sends in the {{ Connection: close }} header, thereby,
signaling to the server to close the TCP socket after the response.
We either need to create a new interface for supporting HTTP pipelining , or
modify the existing {{http::post}} to do so.
This is needed for the Scheduler/Executor library implementations to make sure
"Calls" are sent in order to the master. Currently, in order to do so, we send
in the next request only after we have received a response for an earlier call
that results in degraded performance.
> Support HTTP Pipelining in libprocess (http::post)
> --
>
> Key: MESOS-3332
> URL: https://issues.apache.org/jira/browse/MESOS-3332
> Project: Mesos
> Issue Type: Task
> Components: libprocess
>Reporter: Anand Mazumdar
> Labels: mesosphere, twitter
>
> Currently , {{http::post}} in libprocess, does not support HTTP pipelining.
> Each call as of know sends in the {{Connection: close}} header, thereby,
> signaling to the server to close the TCP socket after the response.
> We either need to create a new interface for supporting HTTP pipelining , or
> modify the existing {{http::post}} to do so.
> This is needed for the Scheduler/Executor library implementations to make
> sure "Calls" are sent in order to the master. Currently, in order to do so,
> we send in the next request only after we have received a response for an
> earlier call that results in degraded performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3339) Implement filtering mechanism for (Scheduler API Events) Testing
Anand Mazumdar created MESOS-3339:
-
Summary: Implement filtering mechanism for (Scheduler API Events)
Testing
Key: MESOS-3339
URL: https://issues.apache.org/jira/browse/MESOS-3339
Project: Mesos
Issue Type: Task
Components: test
Reporter: Anand Mazumdar
Currently, our testing infrastructure does not have a mechanism of
filtering/dropping HTTP events of a particular type from the Scheduler API
response stream. We need a {DROP_HTTP_CALLS} abstraction that can help us to
filter a particular event type.
{code}
// Enqueues all received events into a libprocess queue.
ACTION_P(Enqueue, queue)
{
std::queue events = arg0;
while (!events.empty()) {
// Note that we currently drop HEARTBEATs because most of these tests
// are not designed to deal with heartbeats.
// TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
if (events.front().type() == Event::HEARTBEAT) {
VLOG(1) << "Ignoring HEARTBEAT event";
} else {
queue->put(events.front());
}
events.pop();
}
}
{code}
This helper code is duplicated in at least two places currently, Scheduler
Library/Maintenance Primitives tests. The solution can be as trivial as moving
this helper function to a common test-header or implement a decorator reader
class over {RecordIOReader} having the functionality of filtering events. There
might be other alternative approaches too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3339) Implement filtering mechanism for (Scheduler API Events) Testing
[
https://issues.apache.org/jira/browse/MESOS-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3339:
--
Description:
Currently, our testing infrastructure does not have a mechanism of
filtering/dropping HTTP events of a particular type from the Scheduler API
response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us to
filter a particular event type.
{code}
// Enqueues all received events into a libprocess queue.
ACTION_P(Enqueue, queue)
{
std::queue events = arg0;
while (!events.empty()) {
// Note that we currently drop HEARTBEATs because most of these tests
// are not designed to deal with heartbeats.
// TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
if (events.front().type() == Event::HEARTBEAT) {
VLOG(1) << "Ignoring HEARTBEAT event";
} else {
queue->put(events.front());
}
events.pop();
}
}
{code}
This helper code is duplicated in at least two places currently, Scheduler
Library/Maintenance Primitives tests. The solution can be as trivial as moving
this helper function to a common test-header or implement a decorator reader
class over {{RecordIOReader}} having the functionality of filtering events.
There might be other alternative approaches too.
was:
Currently, our testing infrastructure does not have a mechanism of
filtering/dropping HTTP events of a particular type from the Scheduler API
response stream. We need a {DROP_HTTP_CALLS} abstraction that can help us to
filter a particular event type.
{code}
// Enqueues all received events into a libprocess queue.
ACTION_P(Enqueue, queue)
{
std::queue events = arg0;
while (!events.empty()) {
// Note that we currently drop HEARTBEATs because most of these tests
// are not designed to deal with heartbeats.
// TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
if (events.front().type() == Event::HEARTBEAT) {
VLOG(1) << "Ignoring HEARTBEAT event";
} else {
queue->put(events.front());
}
events.pop();
}
}
{code}
This helper code is duplicated in at least two places currently, Scheduler
Library/Maintenance Primitives tests. The solution can be as trivial as moving
this helper function to a common test-header or implement a decorator reader
class over {RecordIOReader} having the functionality of filtering events. There
might be other alternative approaches too.
> Implement filtering mechanism for (Scheduler API Events) Testing
>
>
> Key: MESOS-3339
> URL: https://issues.apache.org/jira/browse/MESOS-3339
> Project: Mesos
> Issue Type: Task
> Components: test
>Reporter: Anand Mazumdar
>
> Currently, our testing infrastructure does not have a mechanism of
> filtering/dropping HTTP events of a particular type from the Scheduler API
> response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us
> to filter a particular event type.
> {code}
> // Enqueues all received events into a libprocess queue.
> ACTION_P(Enqueue, queue)
> {
> std::queue events = arg0;
> while (!events.empty()) {
> // Note that we currently drop HEARTBEATs because most of these tests
> // are not designed to deal with heartbeats.
> // TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
> if (events.front().type() == Event::HEARTBEAT) {
> VLOG(1) << "Ignoring HEARTBEAT event";
> } else {
> queue->put(events.front());
> }
> events.pop();
> }
> }
> {code}
> This helper code is duplicated in at least two places currently, Scheduler
> Library/Maintenance Primitives tests. The solution can be as trivial as
> moving this helper function to a common test-header or implement a decorator
> reader class over {{RecordIOReader}} having the functionality of filtering
> events. There might be other alternative approaches too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3339) Implement filtering mechanism for (Scheduler API Events) Testing
[
https://issues.apache.org/jira/browse/MESOS-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3339:
--
Description:
Currently, our testing infrastructure does not have a mechanism of
filtering/dropping HTTP events of a particular type from the Scheduler API
response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us to
filter a particular event type.
{code}
// Enqueues all received events into a libprocess queue.
ACTION_P(Enqueue, queue)
{
std::queue events = arg0;
while (!events.empty()) {
// Note that we currently drop HEARTBEATs because most of these tests
// are not designed to deal with heartbeats.
// TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
if (events.front().type() == Event::HEARTBEAT) {
VLOG(1) << "Ignoring HEARTBEAT event";
} else {
queue->put(events.front());
}
events.pop();
}
}
{code}
This helper code is duplicated in at least two places currently, Scheduler
Library/Maintenance Primitives tests.
- The solution can be as trivial as moving this helper function to a common
test-header
- Implement a decorator reader class over {{RecordIOReader}} having the
functionality of filtering events.
- Implement a {{DROP_HTTP_CALLS}} similar to what we do for other protobufs via
{{DROP_CALLS}}.
was:
Currently, our testing infrastructure does not have a mechanism of
filtering/dropping HTTP events of a particular type from the Scheduler API
response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us to
filter a particular event type.
{code}
// Enqueues all received events into a libprocess queue.
ACTION_P(Enqueue, queue)
{
std::queue events = arg0;
while (!events.empty()) {
// Note that we currently drop HEARTBEATs because most of these tests
// are not designed to deal with heartbeats.
// TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
if (events.front().type() == Event::HEARTBEAT) {
VLOG(1) << "Ignoring HEARTBEAT event";
} else {
queue->put(events.front());
}
events.pop();
}
}
{code}
This helper code is duplicated in at least two places currently, Scheduler
Library/Maintenance Primitives tests. The solution can be as trivial as moving
this helper function to a common test-header or implement a decorator reader
class over {{RecordIOReader}} having the functionality of filtering events.
There might be other alternative approaches too.
> Implement filtering mechanism for (Scheduler API Events) Testing
>
>
> Key: MESOS-3339
> URL: https://issues.apache.org/jira/browse/MESOS-3339
> Project: Mesos
> Issue Type: Task
> Components: test
>Reporter: Anand Mazumdar
>
> Currently, our testing infrastructure does not have a mechanism of
> filtering/dropping HTTP events of a particular type from the Scheduler API
> response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us
> to filter a particular event type.
> {code}
> // Enqueues all received events into a libprocess queue.
> ACTION_P(Enqueue, queue)
> {
> std::queue events = arg0;
> while (!events.empty()) {
> // Note that we currently drop HEARTBEATs because most of these tests
> // are not designed to deal with heartbeats.
> // TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
> if (events.front().type() == Event::HEARTBEAT) {
> VLOG(1) << "Ignoring HEARTBEAT event";
> } else {
> queue->put(events.front());
> }
> events.pop();
> }
> }
> {code}
> This helper code is duplicated in at least two places currently, Scheduler
> Library/Maintenance Primitives tests.
> - The solution can be as trivial as moving this helper function to a common
> test-header
> - Implement a decorator reader class over {{RecordIOReader}} having the
> functionality of filtering events.
> - Implement a {{DROP_HTTP_CALLS}} similar to what we do for other protobufs
> via {{DROP_CALLS}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3339) Implement filtering mechanism for (Scheduler API Events) Testing
[
https://issues.apache.org/jira/browse/MESOS-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723789#comment-14723789
]
Anand Mazumdar commented on MESOS-3339:
---
Modified description to list it as a possible approach. I wasn't immediately
sure if this could be done similar to the DROP_CALLS abstraction and hence had
not specified it before.
> Implement filtering mechanism for (Scheduler API Events) Testing
>
>
> Key: MESOS-3339
> URL: https://issues.apache.org/jira/browse/MESOS-3339
> Project: Mesos
> Issue Type: Task
> Components: test
>Reporter: Anand Mazumdar
>
> Currently, our testing infrastructure does not have a mechanism of
> filtering/dropping HTTP events of a particular type from the Scheduler API
> response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us
> to filter a particular event type.
> {code}
> // Enqueues all received events into a libprocess queue.
> ACTION_P(Enqueue, queue)
> {
> std::queue events = arg0;
> while (!events.empty()) {
> // Note that we currently drop HEARTBEATs because most of these tests
> // are not designed to deal with heartbeats.
> // TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
> if (events.front().type() == Event::HEARTBEAT) {
> VLOG(1) << "Ignoring HEARTBEAT event";
> } else {
> queue->put(events.front());
> }
> events.pop();
> }
> }
> {code}
> This helper code is duplicated in at least two places currently, Scheduler
> Library/Maintenance Primitives tests.
> - The solution can be as trivial as moving this helper function to a common
> test-header
> - Implement a decorator reader class over {{RecordIOReader}} having the
> functionality of filtering events.
> - Implement a {{DROP_HTTP_CALLS}} similar to what we do for other protobufs
> via {{DROP_CALLS}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3339) Implement filtering mechanism for (Scheduler API Events) Testing
[
https://issues.apache.org/jira/browse/MESOS-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723789#comment-14723789
]
Anand Mazumdar edited comment on MESOS-3339 at 8/31/15 6:06 PM:
Modified description to list it as a possible approach. It wasn't immediately
obvious to me how this could be done similar to the DROP_CALLS abstraction and
hence had not specified it before.
was (Author: anandmazumdar):
Modified description to list it as a possible approach. I wasn't immediately
sure if this could be done similar to the DROP_CALLS abstraction and hence had
not specified it before.
> Implement filtering mechanism for (Scheduler API Events) Testing
>
>
> Key: MESOS-3339
> URL: https://issues.apache.org/jira/browse/MESOS-3339
> Project: Mesos
> Issue Type: Task
> Components: test
>Reporter: Anand Mazumdar
>
> Currently, our testing infrastructure does not have a mechanism of
> filtering/dropping HTTP events of a particular type from the Scheduler API
> response stream. We need a {{DROP_HTTP_CALLS}} abstraction that can help us
> to filter a particular event type.
> {code}
> // Enqueues all received events into a libprocess queue.
> ACTION_P(Enqueue, queue)
> {
> std::queue events = arg0;
> while (!events.empty()) {
> // Note that we currently drop HEARTBEATs because most of these tests
> // are not designed to deal with heartbeats.
> // TODO(vinod): Implement DROP_HTTP_CALLS that can filter heartbeats.
> if (events.front().type() == Event::HEARTBEAT) {
> VLOG(1) << "Ignoring HEARTBEAT event";
> } else {
> queue->put(events.front());
> }
> events.pop();
> }
> }
> {code}
> This helper code is duplicated in at least two places currently, Scheduler
> Library/Maintenance Primitives tests.
> - The solution can be as trivial as moving this helper function to a common
> test-header
> - Implement a decorator reader class over {{RecordIOReader}} having the
> functionality of filtering events.
> - Implement a {{DROP_HTTP_CALLS}} similar to what we do for other protobufs
> via {{DROP_CALLS}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3343) Rate Limiting functionality for HTTP Frameworks
Anand Mazumdar created MESOS-3343: - Summary: Rate Limiting functionality for HTTP Frameworks Key: MESOS-3343 URL: https://issues.apache.org/jira/browse/MESOS-3343 Project: Mesos Issue Type: Task Reporter: Anand Mazumdar We need to build rate limiting functionality for frameworks connecting via the Scheduler HTTP API similar to the PID based frameworks. Link to the rate-limiting section from design doc: https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#heading=h.kzgdk4d5fmba - This ticket deals with refactoring the existing PID based framework functionality and extend it for HTTP frameworks. - The second part of notifying the framework when rate-limiting is active i.e. returning a status of 429 can be undertook as part of MESOS-1664 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3355) Testing the HTTP V1 API
Anand Mazumdar created MESOS-3355:
-
Summary: Testing the HTTP V1 API
Key: MESOS-3355
URL: https://issues.apache.org/jira/browse/MESOS-3355
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Currently, we don't yet have extensive Fault-Tolerance/Partition/Reconciliation
tests for the HTTP V1 API. The one's in {{tests/fault_tolerance_tests.cpp,
tests/partition_tests.cpp, tests/reconciliation_tests.cpp}} only test the old
PID based workflow.
We need to build the functionality for testing the new V1 API. There can be
various approaches that can help us achieve the objective:
- Implement a new driver speaking HTTP called {{MesosHTTPSchedulerDriver}}
living in {{tests}} folder. We won't expose it to the framework developers and
it would be just used for testing.
- Have an environment variable/constructor argument in the existing
{{MesosSchedulerDriver}} to make it speak HTTP when the relevant flag is set.
- Modify the tests to use the Scheduler library. This can be a bit cumbersome
as the tests have mock expectations etc defined on the old driver callbacks.
All these 3 approaches would still keep the old PID based workflow intact and
we can implement 3 new test files like {{tests/fault_tolerance_http_tests.cpp}}
etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3311) SlaveTest.HTTPSchedulerSlaveRestart
[
https://issues.apache.org/jira/browse/MESOS-3311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3311:
--
Sprint: Mesosphere Sprint 18
Story Points: 2
Labels: flaky-test mesosphere (was: flaky-test)
> SlaveTest.HTTPSchedulerSlaveRestart
> ---
>
> Key: MESOS-3311
> URL: https://issues.apache.org/jira/browse/MESOS-3311
> Project: Mesos
> Issue Type: Bug
> Components: test
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/729/consoleFull
>Reporter: Vinod Kone
>Assignee: Anand Mazumdar
> Labels: flaky-test, mesosphere
> Fix For: 0.24.0
>
>
> Observed on ASF CI
> {code}
> [ RUN ] SlaveTest.HTTPSchedulerSlaveRestart
> Using temporary directory '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA'
> I0825 22:07:36.809872 27610 leveldb.cpp:176] Opened db in 3.751801ms
> I0825 22:07:36.85 27610 leveldb.cpp:183] Compacted db in 1.2194ms
> I0825 22:07:36.811175 27610 leveldb.cpp:198] Created db iterator in 30669ns
> I0825 22:07:36.811197 27610 leveldb.cpp:204] Seeked to beginning of db in
> 7829ns
> I0825 22:07:36.811208 27610 leveldb.cpp:273] Iterated through 0 keys in the
> db in 6017ns
> I0825 22:07:36.811245 27610 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0825 22:07:36.811722 27638 recover.cpp:449] Starting replica recovery
> I0825 22:07:36.811980 27638 recover.cpp:475] Replica is in EMPTY status
> I0825 22:07:36.813033 27641 replica.cpp:641] Replica in EMPTY status received
> a broadcasted recover request
> I0825 22:07:36.813355 27635 recover.cpp:195] Received a recover response from
> a replica in EMPTY status
> I0825 22:07:36.813756 27628 recover.cpp:566] Updating replica status to
> STARTING
> I0825 22:07:36.814434 27636 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 570160ns
> I0825 22:07:36.814471 27636 replica.cpp:323] Persisted replica status to
> STARTING
> I0825 22:07:36.814743 27642 recover.cpp:475] Replica is in STARTING status
> I0825 22:07:36.814965 27638 master.cpp:378] Master
> 20150825-220736-234885548-51219-27610 (09c6504e3a31) started on
> 172.17.0.14:51219
> I0825 22:07:36.814999 27638 master.cpp:380] Flags at startup: --acls=""
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="25secs"
> --registry_strict="true" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/mesos/mesos-0.25.0/_inst/share/mesos/webui"
> --work_dir="/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/master"
> --zk_session_timeout="10secs"
> I0825 22:07:36.815347 27638 master.cpp:425] Master only allowing
> authenticated frameworks to register
> I0825 22:07:36.815371 27638 master.cpp:430] Master only allowing
> authenticated slaves to register
> I0825 22:07:36.815402 27638 credentials.hpp:37] Loading credentials for
> authentication from
> '/tmp/SlaveTest_HTTPSchedulerSlaveRestart_CXyDrA/credentials'
> I0825 22:07:36.815634 27632 replica.cpp:641] Replica in STARTING status
> received a broadcasted recover request
> I0825 22:07:36.815752 27638 master.cpp:469] Using default 'crammd5'
> authenticator
> I0825 22:07:36.815904 27638 master.cpp:506] Authorization enabled
> I0825 22:07:36.815979 27643 recover.cpp:195] Received a recover response from
> a replica in STARTING status
> I0825 22:07:36.816185 27637 whitelist_watcher.cpp:79] No whitelist given
> I0825 22:07:36.816186 27641 hierarchical.hpp:346] Initialized hierarchical
> allocator process
> I0825 22:07:36.816519 27630 recover.cpp:566] Updating replica status to VOTING
> I0825 22:07:36.817258 27639 leveldb.cpp:306] Persisting metadata (8 bytes) to
> leveldb took 475231ns
> I0825 22:07:36.817296 27639 replica.cpp:323] Persisted replica status to
> VOTING
> I0825 22:07:36.817420 27637 master.cpp:1525] The newly elected leader is
> master@172.17.0.14:51219 with id 20150825-220736-234885548-51219-27610
> I0825 22:07:36.817467 27637 master.cpp:1538] Elected as the leading master!
> I0825 22:07:36.817483 27637 master.cpp:1308] Recovering from registrar
[jira] [Commented] (MESOS-3273) EventCall Test Framework is flaky
[
https://issues.apache.org/jira/browse/MESOS-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728577#comment-14728577
]
Anand Mazumdar commented on MESOS-3273:
---
Was able to reproduce this once on my machine. But the issue is not the same as
the one seen on ASF build though as per logs. So we still need to dig around
for what the issue on ASF test-run was.
In my test run, there is a race between the master successfully recovering its
state from registry and the scheduler sending a call. In this case, we just log
the error and leave it upon the framework to retry the call.
{code}
I0902 23:29:28.815498 113774592 leveldb.cpp:438] Reading position from leveldb
took 32us
I0902 23:29:28.826355 136355840 registrar.cpp:344] Successfully fetched the
registry (0B) in 16.811008ms
I0902 23:29:28.826472 136355840 registrar.cpp:443] Applied 1 operations in
35us; attempting to update the 'registry'
I0902 23:29:28.826869 135819264 http.cpp:333] HTTP POST for
/master/api/v1/scheduler from 192.168.29.132:56913
W0902 23:29:28.831881 135282688 scheduler.cpp:381] Received '503 Service
Unavailable' () for SUBSCRIBE
{code}
[~ijimenez] [~vinodkone]
> EventCall Test Framework is flaky
> -
>
> Key: MESOS-3273
> URL: https://issues.apache.org/jira/browse/MESOS-3273
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/705/COMPILER=clang,CONFIGURATION=--verbose,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/consoleFull
>Reporter: Vinod Kone
> Labels: flaky-test, tech-debt, twitter
>
> Observed this on ASF CI. h/t [~haosd...@gmail.com]
> Looks like the HTTP scheduler never sent a SUBSCRIBE request to the master.
> {code}
> [ RUN ] ExamplesTest.EventCallFramework
> Using temporary directory '/tmp/ExamplesTest_EventCallFramework_k4vXkx'
> I0813 19:55:15.643579 26085 exec.cpp:443] Ignoring exited event because the
> driver is aborted!
> Shutting down
> Sending SIGTERM to process tree at pid 26061
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26062
> Shutting down
> Killing the following process trees:
> [
> ]
> Sending SIGTERM to process tree at pid 26063
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26098
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26099
> Killing the following process trees:
> [
> ]
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0813 19:55:17.161726 26100 process.cpp:1012] libprocess is initialized on
> 172.17.2.10:60249 for 16 cpus
> I0813 19:55:17.161888 26100 logging.cpp:177] Logging to STDERR
> I0813 19:55:17.163625 26100 scheduler.cpp:157] Version: 0.24.0
> I0813 19:55:17.175302 26100 leveldb.cpp:176] Opened db in 3.167446ms
> I0813 19:55:17.176393 26100 leveldb.cpp:183] Compacted db in 1.047996ms
> I0813 19:55:17.176496 26100 leveldb.cpp:198] Created db iterator in 77155ns
> I0813 19:55:17.176518 26100 leveldb.cpp:204] Seeked to beginning of db in
> 8429ns
> I0813 19:55:17.176527 26100 leveldb.cpp:273] Iterated through 0 keys in the
> db in 4219ns
> I0813 19:55:17.176708 26100 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0813 19:55:17.178951 26136 recover.cpp:449] Starting replica recovery
> I0813 19:55:17.179934 26136 recover.cpp:475] Replica is in EMPTY status
> I0813 19:55:17.181970 26126 master.cpp:378] Master
> 20150813-195517-167907756-60249-26100 (297daca2d01a) started on
> 172.17.2.10:60249
> I0813 19:55:17.182317 26126 master.cpp:380] Flags at startup:
> --acls="permissive: false
> register_frameworks {
> principals {
> type: SOME
> values: "test-principal"
> }
> roles {
> type: SOME
> values: "*"
> }
> }
> run_tasks {
> principals {
> type: SOME
> values: "test-principal"
> }
> users {
> type: SOME
> values: "mesos"
> }
> }
> " --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="false" --authenticate_slaves="false"
> --authenticators="crammd5"
> --credentials="/tmp/ExamplesTest_EventCallFramework_k4vXkx/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_slave_ping_timeouts="5" --quiet="false"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="5secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/mesos/mesos-0.24.0/src/webu
[jira] [Comment Edited] (MESOS-3273) EventCall Test Framework is flaky
[
https://issues.apache.org/jira/browse/MESOS-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728577#comment-14728577
]
Anand Mazumdar edited comment on MESOS-3273 at 9/3/15 6:59 AM:
---
Was able to reproduce this once on my machine. But the issue is not the same as
the one seen on ASF build though as per logs. So we still need to dig around
for what the issue on ASF test-run was.
In my test run, there is a race between the master successfully recovering its
state from registry and the scheduler sending a call. In this case, we just log
the error and leave it upon the framework to retry the call. This happened only
because I was running in a debugger in a loop and there would have been state
left in Master across various test invocations leading to the time in Registry
Recovery. So , I won't worry too much about this occurrence.
{code}
I0902 23:29:28.815498 113774592 leveldb.cpp:438] Reading position from leveldb
took 32us
I0902 23:29:28.826355 136355840 registrar.cpp:344] Successfully fetched the
registry (0B) in 16.811008ms
I0902 23:29:28.826472 136355840 registrar.cpp:443] Applied 1 operations in
35us; attempting to update the 'registry'
I0902 23:29:28.826869 135819264 http.cpp:333] HTTP POST for
/master/api/v1/scheduler from 192.168.29.132:56913
W0902 23:29:28.831881 135282688 scheduler.cpp:381] Received '503 Service
Unavailable' () for SUBSCRIBE
{code}
[~ijimenez] [~vinodkone]
was (Author: anandmazumdar):
Was able to reproduce this once on my machine. But the issue is not the same as
the one seen on ASF build though as per logs. So we still need to dig around
for what the issue on ASF test-run was.
In my test run, there is a race between the master successfully recovering its
state from registry and the scheduler sending a call. In this case, we just log
the error and leave it upon the framework to retry the call.
{code}
I0902 23:29:28.815498 113774592 leveldb.cpp:438] Reading position from leveldb
took 32us
I0902 23:29:28.826355 136355840 registrar.cpp:344] Successfully fetched the
registry (0B) in 16.811008ms
I0902 23:29:28.826472 136355840 registrar.cpp:443] Applied 1 operations in
35us; attempting to update the 'registry'
I0902 23:29:28.826869 135819264 http.cpp:333] HTTP POST for
/master/api/v1/scheduler from 192.168.29.132:56913
W0902 23:29:28.831881 135282688 scheduler.cpp:381] Received '503 Service
Unavailable' () for SUBSCRIBE
{code}
[~ijimenez] [~vinodkone]
> EventCall Test Framework is flaky
> -
>
> Key: MESOS-3273
> URL: https://issues.apache.org/jira/browse/MESOS-3273
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment:
> https://builds.apache.org/job/Mesos/705/COMPILER=clang,CONFIGURATION=--verbose,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/consoleFull
>Reporter: Vinod Kone
> Labels: flaky-test, tech-debt, twitter
>
> Observed this on ASF CI. h/t [~haosd...@gmail.com]
> Looks like the HTTP scheduler never sent a SUBSCRIBE request to the master.
> {code}
> [ RUN ] ExamplesTest.EventCallFramework
> Using temporary directory '/tmp/ExamplesTest_EventCallFramework_k4vXkx'
> I0813 19:55:15.643579 26085 exec.cpp:443] Ignoring exited event because the
> driver is aborted!
> Shutting down
> Sending SIGTERM to process tree at pid 26061
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26062
> Shutting down
> Killing the following process trees:
> [
> ]
> Sending SIGTERM to process tree at pid 26063
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26098
> Killing the following process trees:
> [
> ]
> Shutting down
> Sending SIGTERM to process tree at pid 26099
> Killing the following process trees:
> [
> ]
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0813 19:55:17.161726 26100 process.cpp:1012] libprocess is initialized on
> 172.17.2.10:60249 for 16 cpus
> I0813 19:55:17.161888 26100 logging.cpp:177] Logging to STDERR
> I0813 19:55:17.163625 26100 scheduler.cpp:157] Version: 0.24.0
> I0813 19:55:17.175302 26100 leveldb.cpp:176] Opened db in 3.167446ms
> I0813 19:55:17.176393 26100 leveldb.cpp:183] Compacted db in 1.047996ms
> I0813 19:55:17.176496 26100 leveldb.cpp:198] Created db iterator in 77155ns
> I0813 19:55:17.176518 26100 leveldb.cpp:204] Seeked to beginning of db in
> 8429ns
> I0813 19:55:17.176527 26100 leveldb.cpp:273] Iterated through 0 keys in the
> db in 4219ns
> I0813 19:55:17.176708 26100 replica.cpp:744] Replica recovered with log
> positions 0 -> 0 with 1 holes and 0 unlearned
> I0813 19:55:17.178951 26136 recover.cpp:449] Starting replica recovery
> I0813 19:55:17.179934 26136 recover.cpp:475] Replica is in EMPTY
[jira] [Commented] (MESOS-3410) MesosContainerizerLaunchTest.ROOT_ChangeRootfs is broken/flaky
[
https://issues.apache.org/jira/browse/MESOS-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739739#comment-14739739
]
Anand Mazumdar commented on MESOS-3410:
---
"ldd" is just displaying the contents of the libraries "/usr/bin/stat" was
linked with at compile time. Does "/usr/lib64/libpcre.so.1" actually exist on
your machine ? I am sure it does, but just re-confirming the obvious.
> MesosContainerizerLaunchTest.ROOT_ChangeRootfs is broken/flaky
> --
>
> Key: MESOS-3410
> URL: https://issues.apache.org/jira/browse/MESOS-3410
> Project: Mesos
> Issue Type: Bug
> Components: containerization, test
>Reporter: Kapil Arya
>
> `sudo make check` failed with the following error on my OpenSUSE Tumbleweed
> box (Linux 4.1, gcc 5.1):
> {code}
> [--] 1 test from MesosContainerizerLaunchTest
> [ RUN ] MesosContainerizerLaunchTest.ROOT_ChangeRootfs
> Changing root to
> /tmp/MesosContainerizerLaunchTest_ROOT_ChangeRootfs_4hGV7G/rootfs
> /usr/bin/stat: error while loading shared libraries: libpcre.so.1: cannot
> open shared object file: No such file or directory
> ../../src/tests/containerizer/launch_tests.cpp:143: Failure
> Value of: *(int *) &(status))) & 0xff00) >> 8)
> Actual: 127
> Expected: 0
> [ FAILED ] MesosContainerizerLaunchTest.ROOT_ChangeRootfs (1171 ms)
> {code}
> Here is the output of ldd:
> {code}
> $> ldd /usr/bin/stat
> linux-vdso.so.1 (0x7fffd83fc000)
> libselinux.so.1 => /lib64/libselinux.so.1 (0x7f80748b3000)
> libc.so.6 => /lib64/libc.so.6 (0x7f807450e000)
> libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x7f807429f000)
> libdl.so.2 => /lib64/libdl.so.2 (0x7f807409b000)
> /lib64/ld-linux-x86-64.so.2 (0x5602eb941000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x7f8073e7d000)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Description:
This is the first basic step in ensuring the basic /call functionality:
- Set up the route on the slave for "api/v1/executor" endpoint.
- The endpoint should perform basic header/protobuf validation and return {501
NotImplemented} for now.
- Introduce initial tests in executor_api_tests.cpp that just verify the status
code.
was:
This is the first basic step in ensuring the basic /call functionality:
processing a
POST /call
and returning:
202 if all goes well;
401 if not authorized; and
403 if the request is malformed.
Also , we might need to store some identifier which enables us to reject calls
to /call if the client has not issued a SUBSCRIBE/RESUBSCRIBE Request.
> Slave : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the slave for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {501 NotImplemented} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Description:
This is the first basic step in ensuring the basic /call functionality:
- Set up the route on the slave for "api/v1/executor" endpoint.
- The endpoint should perform basic header/protobuf validation and return {{501
NotImplemented}} for now.
- Introduce initial tests in executor_api_tests.cpp that just verify the status
code.
was:
This is the first basic step in ensuring the basic /call functionality:
- Set up the route on the slave for "api/v1/executor" endpoint.
- The endpoint should perform basic header/protobuf validation and return {501
NotImplemented} for now.
- Introduce initial tests in executor_api_tests.cpp that just verify the status
code.
> Slave : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the slave for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {{501 NotImplemented}} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Agent : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Summary: Agent : Create Basic Functionality to handle /call endpoint (was:
Slave : Create Basic Functionality to handle /call endpoint)
> Agent : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the slave for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {{501 NotImplemented}} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Slave : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Sprint: Mesosphere Sprint 19
> Slave : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the slave for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {{501 NotImplemented}} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Agent : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Description:
This is the first basic step in ensuring the basic /call functionality:
- Set up the route on the agent for "api/v1/executor" endpoint.
- The endpoint should perform basic header/protobuf validation and return {{501
NotImplemented}} for now.
- Introduce initial tests in executor_api_tests.cpp that just verify the status
code.
was:
This is the first basic step in ensuring the basic /call functionality:
- Set up the route on the slave for "api/v1/executor" endpoint.
- The endpoint should perform basic header/protobuf validation and return {{501
NotImplemented}} for now.
- Introduce initial tests in executor_api_tests.cpp that just verify the status
code.
> Agent : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the agent for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {{501 NotImplemented}} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2708: -- Target Version/s: 0.26.0 (was: 1.0.0) > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Isabel Jimenez > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2295) Implement the Call endpoint on Slave
[ https://issues.apache.org/jira/browse/MESOS-2295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2295: -- Target Version/s: 0.26.0 (was: 1.0.0) > Implement the Call endpoint on Slave > > > Key: MESOS-2295 > URL: https://issues.apache.org/jira/browse/MESOS-2295 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2906) Slave : Synchronous Validation for Calls
[ https://issues.apache.org/jira/browse/MESOS-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-2906: -- Target Version/s: 0.26.0 (was: 1.0.0) > Slave : Synchronous Validation for Calls > > > Key: MESOS-2906 > URL: https://issues.apache.org/jira/browse/MESOS-2906 > Project: Mesos > Issue Type: Task >Reporter: Anand Mazumdar >Assignee: Anand Mazumdar > Labels: HTTP, mesosphere > > /call endpoint on the slave will return a 202 accepted code but has to do > some basic validations before. In case of invalidation it will return a 4xx > code. > - We need to create the required infrastructure to validate the request and > then process it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2907) Agent : Create Basic Functionality to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2907:
--
Target Version/s: 0.26.0 (was: 1.0.0)
> Agent : Create Basic Functionality to handle /call endpoint
> ---
>
> Key: MESOS-2907
> URL: https://issues.apache.org/jira/browse/MESOS-2907
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> This is the first basic step in ensuring the basic /call functionality:
> - Set up the route on the agent for "api/v1/executor" endpoint.
> - The endpoint should perform basic header/protobuf validation and return
> {{501 NotImplemented}} for now.
> - Introduce initial tests in executor_api_tests.cpp that just verify the
> status code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2906) Slave : Synchronous Validation for Calls
[
https://issues.apache.org/jira/browse/MESOS-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-2906:
--
Description:
/call endpoint on the slave will return a 202 accepted code but has to do some
basic validations before. In case of invalidation it will return a
{{BadRequest}} back to the client.
- We need to create the required infrastructure to validate the request and
then process it similar to {{src/master/validation.cpp}} in the {{namespace
scheduler}} i.e. check if the protobuf is properly initialized, has the
required attributes set pertaining to the call message etc.
was:
/call endpoint on the slave will return a 202 accepted code but has to do some
basic validations before. In case of invalidation it will return a 4xx code.
- We need to create the required infrastructure to validate the request and
then process it.
> Slave : Synchronous Validation for Calls
>
>
> Key: MESOS-2906
> URL: https://issues.apache.org/jira/browse/MESOS-2906
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: HTTP, mesosphere
>
> /call endpoint on the slave will return a 202 accepted code but has to do
> some basic validations before. In case of invalidation it will return a
> {{BadRequest}} back to the client.
> - We need to create the required infrastructure to validate the request and
> then process it similar to {{src/master/validation.cpp}} in the {{namespace
> scheduler}} i.e. check if the protobuf is properly initialized, has the
> required attributes set pertaining to the call message etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-2296) Implement the Events stream on slave for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-2296: - Assignee: Anand Mazumdar (was: Klaus Ma) > Implement the Events stream on slave for Call endpoint > -- > > Key: MESOS-2296 > URL: https://issues.apache.org/jira/browse/MESOS-2296 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876681#comment-14876681 ] Anand Mazumdar commented on MESOS-2708: --- [~vinodkone] I edited the document further based on your feedback. Modified the Agent Recovery section. Added a Backoff Strategies section as we had discussed. Also, modified the name of a few environment variables so that they can be re-used elsewhere in future. Let me know if you have more comments. > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Isabel Jimenez > Labels: mesosphere > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3476) Refactor Status Update method on Slave to handle HTTP Executors
Anand Mazumdar created MESOS-3476:
-
Summary: Refactor Status Update method on Slave to handle HTTP
Executors
Key: MESOS-3476
URL: https://issues.apache.org/jira/browse/MESOS-3476
Project: Mesos
Issue Type: Bug
Reporter: Anand Mazumdar
Currently, receiving a status update sent from slave to itself , {{runTask}} ,
{{killTask}} and status updates from executors are handled by the
{{Slave::statusUpdate}} method on Slave. The signature of the method is {{void
Slave::statusUpdate(StatusUpdate update, const UPID& pid)}}.
We need to create another overload of it that can also handle HTTP based
executors which the previous PID based function can also call into. The
signature of the new function could be:
{{void Slave::statusUpdate(StatusUpdate update, Executor* executor)}}
The HTTP Executor would also call into this new function via
{{src/slave/http.cpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-3476) Refactor Status Update method on Slave to handle HTTP Executors
[
https://issues.apache.org/jira/browse/MESOS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar reassigned MESOS-3476:
-
Assignee: Anand Mazumdar
> Refactor Status Update method on Slave to handle HTTP Executors
> ---
>
> Key: MESOS-3476
> URL: https://issues.apache.org/jira/browse/MESOS-3476
> Project: Mesos
> Issue Type: Bug
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> Currently, receiving a status update sent from slave to itself , {{runTask}}
> , {{killTask}} and status updates from executors are handled by the
> {{Slave::statusUpdate}} method on Slave. The signature of the method is
> {{void Slave::statusUpdate(StatusUpdate update, const UPID& pid)}}.
> We need to create another overload of it that can also handle HTTP based
> executors which the previous PID based function can also call into. The
> signature of the new function could be:
> {{void Slave::statusUpdate(StatusUpdate update, Executor* executor)}}
> The HTTP Executor would also call into this new function via
> {{src/slave/http.cpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3476) Refactor Status Update method on Slave to handle HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3476:
--
Summary: Refactor Status Update method on Slave to handle HTTP based
Executors (was: Refactor Status Update method on Slave to handle HTTP
Executors)
> Refactor Status Update method on Slave to handle HTTP based Executors
> -
>
> Key: MESOS-3476
> URL: https://issues.apache.org/jira/browse/MESOS-3476
> Project: Mesos
> Issue Type: Bug
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> Currently, receiving a status update sent from slave to itself , {{runTask}}
> , {{killTask}} and status updates from executors are handled by the
> {{Slave::statusUpdate}} method on Slave. The signature of the method is
> {{void Slave::statusUpdate(StatusUpdate update, const UPID& pid)}}.
> We need to create another overload of it that can also handle HTTP based
> executors which the previous PID based function can also call into. The
> signature of the new function could be:
> {{void Slave::statusUpdate(StatusUpdate update, Executor* executor)}}
> The HTTP Executor would also call into this new function via
> {{src/slave/http.cpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3480) Refactor Executor struct in Slave to handle HTTP based executors
Anand Mazumdar created MESOS-3480:
-
Summary: Refactor Executor struct in Slave to handle HTTP based
executors
Key: MESOS-3480
URL: https://issues.apache.org/jira/browse/MESOS-3480
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Assignee: Anand Mazumdar
Fix For: 0.26.0
Currently, the {{struct Executor}} in slave only supports executors connected
via message passing (driver). We should refactor it to add support for HTTP
based Executors similar to what was done for the Scheduler API {{struct
Framework}} in {{src/master/master.hpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3476) Refactor Status Update method on Slave to handle HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3476:
--
Issue Type: Task (was: Bug)
> Refactor Status Update method on Slave to handle HTTP based Executors
> -
>
> Key: MESOS-3476
> URL: https://issues.apache.org/jira/browse/MESOS-3476
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> Currently, receiving a status update sent from slave to itself , {{runTask}}
> , {{killTask}} and status updates from executors are handled by the
> {{Slave::statusUpdate}} method on Slave. The signature of the method is
> {{void Slave::statusUpdate(StatusUpdate update, const UPID& pid)}}.
> We need to create another overload of it that can also handle HTTP based
> executors which the previous PID based function can also call into. The
> signature of the new function could be:
> {{void Slave::statusUpdate(StatusUpdate update, Executor* executor)}}
> The HTTP Executor would also call into this new function via
> {{src/slave/http.cpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3490) Mesos UI fails to represent JSON entities
[ https://issues.apache.org/jira/browse/MESOS-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-3490: -- Description: The Mesos UI is broken, it seems to fail to represent JSON from /state. This may have been introduced with https://reviews.apache.org/r/38028 was: The Mesos UI is broken, it seems to fail to represent JSON from /state. This may got introduced with https://reviews.apache.org/r/38028/. > Mesos UI fails to represent JSON entities > - > > Key: MESOS-3490 > URL: https://issues.apache.org/jira/browse/MESOS-3490 > Project: Mesos > Issue Type: Bug > Components: stout >Reporter: Isabel Jimenez >Assignee: Joseph Wu > Labels: mesosphere > > The Mesos UI is broken, it seems to fail to represent JSON from /state. > This may have been introduced with https://reviews.apache.org/r/38028 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3490) Mesos UI fails to represent JSON entities
[ https://issues.apache.org/jira/browse/MESOS-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-3490: -- Sprint: Mesosphere Sprint 19 > Mesos UI fails to represent JSON entities > - > > Key: MESOS-3490 > URL: https://issues.apache.org/jira/browse/MESOS-3490 > Project: Mesos > Issue Type: Bug > Components: stout >Reporter: Isabel Jimenez >Assignee: Joseph Wu > Labels: mesosphere > > The Mesos UI is broken, it seems to fail to represent JSON from /state. > This may have been introduced with https://reviews.apache.org/r/38028 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3480) Refactor Executor struct in Slave to handle HTTP based executors
[
https://issues.apache.org/jira/browse/MESOS-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3480:
--
Fix Version/s: (was: 0.26.0)
> Refactor Executor struct in Slave to handle HTTP based executors
>
>
> Key: MESOS-3480
> URL: https://issues.apache.org/jira/browse/MESOS-3480
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> Currently, the {{struct Executor}} in slave only supports executors connected
> via message passing (driver). We should refactor it to add support for HTTP
> based Executors similar to what was done for the Scheduler API {{struct
> Framework}} in {{src/master/master.hpp}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3051) performance issues with port ranges comparison
[
https://issues.apache.org/jira/browse/MESOS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14904928#comment-14904928
]
Anand Mazumdar commented on MESOS-3051:
---
[~js84], Can we file a followup JIRA based on comments by [~jieyu] to consider
using {{IntervalSet}} in the near future ? Ignore my comments if one already
exists.
> performance issues with port ranges comparison
> --
>
> Key: MESOS-3051
> URL: https://issues.apache.org/jira/browse/MESOS-3051
> Project: Mesos
> Issue Type: Bug
> Components: allocation
>Affects Versions: 0.22.1
>Reporter: James Peach
>Assignee: Joerg Schad
> Labels: mesosphere
> Fix For: 0.25.0
>
>
> Testing in an environment with lots of frameworks (>200), where the
> frameworks permanently decline resources they don't need. The allocator ends
> up spending a lot of time figuring out whether offers are refused (the code
> path through {{HierarchicalAllocatorProcess::isFiltered()}}.
> In profiling a synthetic benchmark, it turns out that comparing port ranges
> is very expensive, involving many temporary allocations. 61% of
> Resources::contains() run time is in operator -= (Resource). 35% of
> Resources::contains() run time is in Resources::_contains().
> The heaviest call chain through {{Resources::_contains}} is:
> {code}
> Running Time Self (ms) Symbol Name
> 7237.0ms 35.5% 4.0
> mesos::Resources::_contains(mesos::Resource const&) const
> 7200.0ms 35.3% 1.0 mesos::contains(mesos::Resource
> const&, mesos::Resource const&)
> 7133.0ms 35.0%121.0
> mesos::operator<=(mesos::Value_Ranges const&, mesos::Value_Ranges const&)
> 6319.0ms 31.0% 7.0
> mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Ranges const&)
> 6240.0ms 30.6%161.0
> mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
> 1867.0ms9.1% 25.0 mesos::Value_Ranges::add_range()
> 1694.0ms8.3% 4.0
> mesos::Value_Ranges::~Value_Ranges()
> 1495.0ms7.3% 16.0
> mesos::Value_Ranges::operator=(mesos::Value_Ranges const&)
> 445.0ms2.1% 94.0
> mesos::Value_Range::MergeFrom(mesos::Value_Range const&)
> 154.0ms0.7% 24.0 mesos::Value_Ranges::range(int)
> const
> 103.0ms0.5% 24.0
> mesos::Value_Ranges::range_size() const
> 95.0ms0.4% 2.0
> mesos::Value_Range::Value_Range(mesos::Value_Range const&)
> 59.0ms0.2% 4.0
> mesos::Value_Ranges::Value_Ranges()
> 50.0ms0.2% 50.0 mesos::Value_Range::begin()
> const
> 28.0ms0.1% 28.0 mesos::Value_Range::end() const
> 26.0ms0.1% 0.0
> mesos::Value_Range::~Value_Range()
> {code}
> mesos::coalesce(Value_Ranges) gets done a lot and ends up being really
> expensive. The heaviest parts of the inverted call chain are:
> {code}
> Running Time Self (ms) Symbol Name
> 3209.0ms 15.7% 3209.0 mesos::Value_Range::~Value_Range()
> 3209.0ms 15.7% 0.0
> google::protobuf::internal::GenericTypeHandler::Delete(mesos::Value_Range*)
> 3209.0ms 15.7% 0.0 void
> google::protobuf::internal::RepeatedPtrFieldBase::Destroy::TypeHandler>()
> 3209.0ms 15.7% 0.0
> google::protobuf::RepeatedPtrField::~RepeatedPtrField()
> 3209.0ms 15.7% 0.0
> google::protobuf::RepeatedPtrField::~RepeatedPtrField()
> 3209.0ms 15.7% 0.0
> mesos::Value_Ranges::~Value_Ranges()
> 3209.0ms 15.7% 0.0
> mesos::Value_Ranges::~Value_Ranges()
> 2441.0ms 11.9% 0.0
> mesos::coalesce(mesos::Value_Ranges*, mesos::Value_Range const&)
> 452.0ms2.2% 0.0
> mesos::remove(mesos::Value_Ranges*, mesos::Value_Range const&)
> 169.0ms0.8% 0.0
> mesos::operator<=(mesos::Value_Ranges const&, mesos::Value_Ranges const&)
> 82.0ms0.4% 0.0
> mesos::operator-=(mesos::Value_Ranges&, mesos::Value_Ranges const&)
> 65.0ms0.3% 0.0
> mesos::Value_Ranges::~Value_Ranges()
> 2541.0ms 12.4% 2541.0
> google::protobuf::internal::GenericTypeHandler::New()
> 2541.0ms 12.4% 0.0
> google::protobuf::RepeatedPtrField::TypeHandler::Type*
> google::protobuf::internal::RepeatedPtrFieldBase::Add::TypeHandler>()
> 2305.0ms 11.3% 0.0
> google::protobuf::RepeatedPtrField::Add()
> 2305.0ms 1
[jira] [Created] (MESOS-3515) Support Subscribe Call for HTTP based Executors
Anand Mazumdar created MESOS-3515:
-
Summary: Support Subscribe Call for HTTP based Executors
Key: MESOS-3515
URL: https://issues.apache.org/jira/browse/MESOS-3515
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Assignee: Anand Mazumdar
We need to add a {{subscribe(...)}} method in {{src/slave/slave.cpp}} to
introduce the ability for HTTP based executors to subscribe and then receive
events on the persistent HTTP connection. Most of the functionality needed
would be similar to {{Master::subscribe}} in {{src/master/master.cpp}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3520) Add an abstraction to manage the life cycle of file descriptors
[ https://issues.apache.org/jira/browse/MESOS-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-3520: -- Summary: Add an abstraction to manage the life cycle of file descriptors (was: Added an abstraction to manage the life cycle of file descriptors) > Add an abstraction to manage the life cycle of file descriptors > --- > > Key: MESOS-3520 > URL: https://issues.apache.org/jira/browse/MESOS-3520 > Project: Mesos > Issue Type: Bug >Reporter: Chi Zhang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3515) Support Subscribe Call for HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3515:
--
Sprint: Mesosphere Sprint 19
Story Points: 3
> Support Subscribe Call for HTTP based Executors
> ---
>
> Key: MESOS-3515
> URL: https://issues.apache.org/jira/browse/MESOS-3515
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> We need to add a {{subscribe(...)}} method in {{src/slave/slave.cpp}} to
> introduce the ability for HTTP based executors to subscribe and then receive
> events on the persistent HTTP connection. Most of the functionality needed
> would be similar to {{Master::subscribe}} in {{src/master/master.cpp}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3550) Create a Executor Library based on the new Executor HTTP API
Anand Mazumdar created MESOS-3550:
-
Summary: Create a Executor Library based on the new Executor HTTP
API
Key: MESOS-3550
URL: https://issues.apache.org/jira/browse/MESOS-3550
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Similar to the Scheduler Library {{src/scheduler/scheduler.cpp}} , we would
need a Executor Library that speaks the new Executor HTTP API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3515) Support Subscribe Call for HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3515:
--
Shepherd: Vinod Kone
> Support Subscribe Call for HTTP based Executors
> ---
>
> Key: MESOS-3515
> URL: https://issues.apache.org/jira/browse/MESOS-3515
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> We need to add a {{subscribe(...)}} method in {{src/slave/slave.cpp}} to
> introduce the ability for HTTP based executors to subscribe and then receive
> events on the persistent HTTP connection. Most of the functionality needed
> would be similar to {{Master::subscribe}} in {{src/master/master.cpp}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3558) Make the CommandExecutor use the Executor Library speaking HTTP
Anand Mazumdar created MESOS-3558:
-
Summary: Make the CommandExecutor use the Executor Library
speaking HTTP
Key: MESOS-3558
URL: https://issues.apache.org/jira/browse/MESOS-3558
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Instead of using the {{MesosExecutorDriver}} , we should make the
{{CommandExecutor}} in {{src/launcher/executor.cpp}} use the new Executor HTTP
Library that we create in {{MESOS-3550}}.
This would act as a good validation of the {{HTTP API}} implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3559) Make the Command Scheduler use the HTTP Scheduler Library
Anand Mazumdar created MESOS-3559:
-
Summary: Make the Command Scheduler use the HTTP Scheduler Library
Key: MESOS-3559
URL: https://issues.apache.org/jira/browse/MESOS-3559
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
We should make the Command Scheduler in {{src/cli/executor.cpp}} use the
Scheduler Library {{src/scheduler/scheduler.cpp}} instead of the Scheduler
Driver.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939194#comment-14939194
]
Anand Mazumdar commented on MESOS-3562:
---
[~BenWhitehead] There seems to be some confusion here. Comments Inline.
> This isn't really standard chunks though, there are chunks within chunks and
> the configuration of the client would have to know that.
Can you elaborate a bit more on what do you mean by chunks between chunks here
? We strictly adhere to the standard chunk encoding format defined in RFC 2616.
The only difference here is that the {{data}} in chunks itself is encoded in
{{RecordIO}} format.
> What is the motivation behind using recordio format ?
We wanted a way to delimit two events for JSON/Protobuf responses and RecordIO
format allowed us to do that. We could have gone away with RecordIO for JSON
though by just delimiting on {{\n}} but that would have made it inconsistent in
behavior when compared to Protobuf Responses.
> If standard encoding were used then every HTTP client would already have the
> necessary understanding to know how to deal with the chunks.
We use standard chunk encoding as defined in RFC. What do you mean here ?
> Where is the specification for what recordio format is? I have not been able
> to find anything online.
We should add more information on this in our docs. For now, till we do that,
here is a brief description on what the format looks like:
{code}
5\n
hello
6\n
world!
{code}
Ideally, whatever client you are using should do the de-chunking for you. You
should get this back from the client i.e. just the {{RecordIO}} encoded data.
{code}
104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
{code}
cc'ing [~bmahler] If I missed anything.
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to
[jira] [Created] (MESOS-3566) Add a section to the Scheduler HTTP API docs around RecordIO specification
Anand Mazumdar created MESOS-3566:
-
Summary: Add a section to the Scheduler HTTP API docs around
RecordIO specification
Key: MESOS-3566
URL: https://issues.apache.org/jira/browse/MESOS-3566
Project: Mesos
Issue Type: Improvement
Reporter: Anand Mazumdar
Since the {{RecordIO}} format is not that widely used, searching for it online
does not offer much help.
- It would be good if we can add to the docs, a small section on its
specification for framework developers.
- Bonus points, if we can have a simple code snippet in C++/Java on reading a
{{RecordIO}} response to help developers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3570) Make Scheduler Library use HTTP Pipelining Abstraction in Libprocess
Anand Mazumdar created MESOS-3570:
-
Summary: Make Scheduler Library use HTTP Pipelining Abstraction in
Libprocess
Key: MESOS-3570
URL: https://issues.apache.org/jira/browse/MESOS-3570
Project: Mesos
Issue Type: Bug
Reporter: Anand Mazumdar
Currently, the scheduler library sends calls in order by chaining them and
sending them only when it has received a response for the earlier call. This
was done because there was no HTTP Pipelining abstraction in Libprocess
{{process::post}}.
However once {{MESOS-3332}} is resolved, we should be now able to use the new
abstraction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3575) V1 API java protos are not generated
[ https://issues.apache.org/jira/browse/MESOS-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-3575: -- Summary: V1 API java protos are not generated (was: HTTP V1 API java protos are not generated) > V1 API java protos are not generated > > > Key: MESOS-3575 > URL: https://issues.apache.org/jira/browse/MESOS-3575 > Project: Mesos > Issue Type: Bug > Components: HTTP API >Reporter: Joris Van Remoortere >Priority: Blocker > > The java protos for the HTTP C1 api should be generated according to the > Makefile; however, they do not show up in the generated build directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3575) V1 API java protos are not generated
[ https://issues.apache.org/jira/browse/MESOS-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar updated MESOS-3575: -- Description: The java protos for the V1 api should be generated according to the Makefile; however, they do not show up in the generated build directory. (was: The java protos for the HTTP C1 api should be generated according to the Makefile; however, they do not show up in the generated build directory.) > V1 API java protos are not generated > > > Key: MESOS-3575 > URL: https://issues.apache.org/jira/browse/MESOS-3575 > Project: Mesos > Issue Type: Bug > Components: HTTP API >Reporter: Joris Van Remoortere >Priority: Blocker > > The java protos for the V1 api should be generated according to the Makefile; > however, they do not show up in the generated build directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3575) V1 API java protos are not generated
[ https://issues.apache.org/jira/browse/MESOS-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Mazumdar reassigned MESOS-3575: - Assignee: Anand Mazumdar > V1 API java protos are not generated > > > Key: MESOS-3575 > URL: https://issues.apache.org/jira/browse/MESOS-3575 > Project: Mesos > Issue Type: Bug > Components: HTTP API >Reporter: Joris Van Remoortere >Assignee: Anand Mazumdar >Priority: Blocker > > The java protos for the V1 api should be generated according to the Makefile; > however, they do not show up in the generated build directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3575) V1 API java protos are not generated
[ https://issues.apache.org/jira/browse/MESOS-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941689#comment-14941689 ] Anand Mazumdar commented on MESOS-3575: --- The scope of this JIRA should be to just ensure that they get generated in the build folder. Hence , it should not be a blocker for 0.25.0 since we don't ship those artifacts. MESOS-3524 talks about how we would like to distribute the protobufs eventually via Maven/other channels. [~jvanremoortere] What do you think ? > V1 API java protos are not generated > > > Key: MESOS-3575 > URL: https://issues.apache.org/jira/browse/MESOS-3575 > Project: Mesos > Issue Type: Bug > Components: HTTP API >Reporter: Joris Van Remoortere >Assignee: Anand Mazumdar >Priority: Blocker > > The java protos for the V1 api should be generated according to the Makefile; > however, they do not show up in the generated build directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3575) V1 API java/python protos are not generated
[ https://issues.apache.org/jira/browse/MESOS-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941895#comment-14941895 ] Anand Mazumdar commented on MESOS-3575: --- I submitted a patch https://reviews.apache.org/r/38967 for stuffing the V1 Protobufs into the existing Mesos JAR for the time being for anyone to start playing with the new API. For python, the change would be much more involved. Our python infrastructure in the Makefile needs some serious re-work. It's a mess to say the least. I would file another JIRA for the issues around it that I noticed while trying to make this change. > V1 API java/python protos are not generated > --- > > Key: MESOS-3575 > URL: https://issues.apache.org/jira/browse/MESOS-3575 > Project: Mesos > Issue Type: Bug > Components: HTTP API >Reporter: Joris Van Remoortere >Assignee: Anand Mazumdar >Priority: Blocker > > The java/python protos for the V1 api should be generated according to the > Makefile; however, they do not show up in the generated build directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3577) OversubscriptionTest.UpdateAllocatorOnSchedulerFailover is flaky
Anand Mazumdar created MESOS-3577:
-
Summary: OversubscriptionTest.UpdateAllocatorOnSchedulerFailover
is flaky
Key: MESOS-3577
URL: https://issues.apache.org/jira/browse/MESOS-3577
Project: Mesos
Issue Type: Bug
Reporter: Anand Mazumdar
Showed up on ASF
https://builds.apache.org/job/Mesos/890/COMPILER=gcc,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=centos:7,label_exp=docker%7C%7CHadoop/consoleFull
{code}
[ RUN ] OversubscriptionTest.UpdateAllocatorOnSchedulerFailover
Using temporary directory
'/tmp/OversubscriptionTest_UpdateAllocatorOnSchedulerFailover_y5LK6v'
I1003 20:29:03.367100 31549 leveldb.cpp:176] Opened db in 2.322276ms
I1003 20:29:03.368028 31549 leveldb.cpp:183] Compacted db in 888247ns
I1003 20:29:03.368093 31549 leveldb.cpp:198] Created db iterator in 22626ns
I1003 20:29:03.368108 31549 leveldb.cpp:204] Seeked to beginning of db in 1842ns
I1003 20:29:03.368115 31549 leveldb.cpp:273] Iterated through 0 keys in the db
in 395ns
I1003 20:29:03.368165 31549 replica.cpp:744] Replica recovered with log
positions 0 -> 0 with 1 holes and 0 unlearned
I1003 20:29:03.368722 31575 recover.cpp:449] Starting replica recovery
I1003 20:29:03.369118 31575 recover.cpp:475] Replica is in EMPTY status
I1003 20:29:03.370707 31572 replica.cpp:641] Replica in EMPTY status received a
broadcasted recover request
I1003 20:29:03.371100 31572 master.cpp:376] Master
d4ff5e08-2202-4f3b-8fb2-5515adf9a97e (9efc27440ed0) started on 172.17.5.73:38504
I1003 20:29:03.371124 31572 master.cpp:378] Flags at startup: --acls=""
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
--authorizers="local"
--credentials="/tmp/OversubscriptionTest_UpdateAllocatorOnSchedulerFailover_y5LK6v/credentials"
--framework_sorter="drf" --help="false" --hostname_lookup="true"
--initialize_driver_logging="true" --log_auto_initialize="true"
--logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5"
--quiet="false" --recovery_slave_removal_limit="100%"
--registry="replicated_log" --registry_fetch_timeout="1mins"
--registry_store_timeout="25secs" --registry_strict="true"
--root_submissions="true" --slave_ping_timeout="15secs"
--slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
--webui_dir="/mesos/mesos-0.26.0/_inst/share/mesos/webui"
--work_dir="/tmp/OversubscriptionTest_UpdateAllocatorOnSchedulerFailover_y5LK6v/master"
--zk_session_timeout="10secs"
I1003 20:29:03.371477 31572 master.cpp:423] Master only allowing authenticated
frameworks to register
I1003 20:29:03.371496 31572 master.cpp:428] Master only allowing authenticated
slaves to register
I1003 20:29:03.371510 31572 credentials.hpp:37] Loading credentials for
authentication from
'/tmp/OversubscriptionTest_UpdateAllocatorOnSchedulerFailover_y5LK6v/credentials'
I1003 20:29:03.371841 31572 master.cpp:467] Using default 'crammd5'
authenticator
I1003 20:29:03.371989 31572 master.cpp:504] Authorization enabled
I1003 20:29:03.372009 31580 recover.cpp:195] Received a recover response from a
replica in EMPTY status
I1003 20:29:03.372231 31568 hierarchical.hpp:468] Initialized hierarchical
allocator process
I1003 20:29:03.372349 31579 whitelist_watcher.cpp:79] No whitelist given
I1003 20:29:03.373409 31572 recover.cpp:566] Updating replica status to STARTING
I1003 20:29:03.373558 31576 master.cpp:1603] The newly elected leader is
master@172.17.5.73:38504 with id d4ff5e08-2202-4f3b-8fb2-5515adf9a97e
I1003 20:29:03.373670 31576 master.cpp:1616] Elected as the leading master!
I1003 20:29:03.373775 31576 master.cpp:1376] Recovering from registrar
I1003 20:29:03.374174 31579 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 611973ns
I1003 20:29:03.374233 31581 registrar.cpp:309] Recovering registrar
I1003 20:29:03.374248 31579 replica.cpp:323] Persisted replica status to
STARTING
I1003 20:29:03.374455 31579 recover.cpp:475] Replica is in STARTING status
I1003 20:29:03.375416 31576 replica.cpp:641] Replica in STARTING status
received a broadcasted recover request
I1003 20:29:03.375880 31575 recover.cpp:195] Received a recover response from a
replica in STARTING status
I1003 20:29:03.376230 31576 recover.cpp:566] Updating replica status to VOTING
I1003 20:29:03.376729 31580 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 370830ns
I1003 20:29:03.376752 31580 replica.cpp:323] Persisted replica status to VOTING
I1003 20:29:03.376893 31580 recover.cpp:580] Successfully joined the Paxos group
I1003 20:29:03.377115 31580 recover.cpp:464] Recover process terminated
I1003 20:29:03.377531 31569 log.cpp:661] Attempting to start the writer
I1003 20:29:03.378665 31583 replica.cpp:477] Replica received implicit promise
request with proposal 1
I1003 20:29:03.379005 31583 leveldb.cpp:306] Persisting metadata
[jira] [Created] (MESOS-3578) ProvisionerDockerLocalStoreTest.MetadataManagerInitialization is flaky
Anand Mazumdar created MESOS-3578:
-
Summary:
ProvisionerDockerLocalStoreTest.MetadataManagerInitialization is flaky
Key: MESOS-3578
URL: https://issues.apache.org/jira/browse/MESOS-3578
Project: Mesos
Issue Type: Bug
Components: containerization
Reporter: Anand Mazumdar
Showed up on ASF CI:
https://builds.apache.org/job/Mesos/881/COMPILER=gcc,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/consoleFull
{code}
[ RUN ] ProvisionerDockerLocalStoreTest.MetadataManagerInitialization
Using temporary directory
'/tmp/ProvisionerDockerLocalStoreTest_MetadataManagerInitialization_9ynmgE'
I0929 02:36:44.066397 30457 local_puller.cpp:127] Untarring image from
'/tmp/ProvisionerDockerLocalStoreTest_MetadataManagerInitialization_9ynmgE/store/staging/aZND7C'
to
'/tmp/ProvisionerDockerLocalStoreTest_MetadataManagerInitialization_9ynmgE/images/abc:latest.tar'
../../src/tests/containerizer/provisioner_docker_tests.cpp:843: Failure
(layers).failure(): Collect failed: Untar failed with exit code: exited with
status 2
[ FAILED ] ProvisionerDockerLocalStoreTest.MetadataManagerInitialization (181
ms)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3579) FetcherCacheTest.LocalUncachedExtract is flaky
Anand Mazumdar created MESOS-3579:
-
Summary: FetcherCacheTest.LocalUncachedExtract is flaky
Key: MESOS-3579
URL: https://issues.apache.org/jira/browse/MESOS-3579
Project: Mesos
Issue Type: Bug
Components: fetcher, test
Reporter: Anand Mazumdar
>From ASF CI:
https://builds.apache.org/job/Mesos/866/COMPILER=clang,CONFIGURATION=--verbose,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/console
{code}
[ RUN ] FetcherCacheTest.LocalUncachedExtract
Using temporary directory '/tmp/FetcherCacheTest_LocalUncachedExtract_jHBfeA'
I0925 19:15:39.541198 27410 leveldb.cpp:176] Opened db in 3.43934ms
I0925 19:15:39.542362 27410 leveldb.cpp:183] Compacted db in 1.136184ms
I0925 19:15:39.542428 27410 leveldb.cpp:198] Created db iterator in 35866ns
I0925 19:15:39.542448 27410 leveldb.cpp:204] Seeked to beginning of db in 8807ns
I0925 19:15:39.542459 27410 leveldb.cpp:273] Iterated through 0 keys in the db
in 6325ns
I0925 19:15:39.542505 27410 replica.cpp:744] Replica recovered with log
positions 0 -> 0 with 1 holes and 0 unlearned
I0925 19:15:39.543143 27438 recover.cpp:449] Starting replica recovery
I0925 19:15:39.543393 27438 recover.cpp:475] Replica is in EMPTY status
I0925 19:15:39.544373 27436 replica.cpp:641] Replica in EMPTY status received a
broadcasted recover request
I0925 19:15:39.544791 27433 recover.cpp:195] Received a recover response from a
replica in EMPTY status
I0925 19:15:39.545284 27433 recover.cpp:566] Updating replica status to STARTING
I0925 19:15:39.546155 27436 master.cpp:376] Master
c8bf1c95-50f4-4832-a570-c560f0b466ae (f57fd4291168) started on
172.17.1.195:41781
I0925 19:15:39.546257 27433 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 747249ns
I0925 19:15:39.546288 27433 replica.cpp:323] Persisted replica status to
STARTING
I0925 19:15:39.546483 27434 recover.cpp:475] Replica is in STARTING status
I0925 19:15:39.546187 27436 master.cpp:378] Flags at startup: --acls=""
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
--authorizers="local"
--credentials="/tmp/FetcherCacheTest_LocalUncachedExtract_jHBfeA/credentials"
--framework_sorter="drf" --help="false" --hostname_lookup="true"
--initialize_driver_logging="true" --log_auto_initialize="true"
--logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5"
--quiet="false" --recovery_slave_removal_limit="100%"
--registry="replicated_log" --registry_fetch_timeout="1mins"
--registry_store_timeout="25secs" --registry_strict="true"
--root_submissions="true" --slave_ping_timeout="15secs"
--slave_reregister_timeout="10mins" --user_sorter="drf" --version="false"
--webui_dir="/mesos/mesos-0.26.0/_inst/share/mesos/webui"
--work_dir="/tmp/FetcherCacheTest_LocalUncachedExtract_jHBfeA/master"
--zk_session_timeout="10secs"
I0925 19:15:39.546567 27436 master.cpp:423] Master only allowing authenticated
frameworks to register
I0925 19:15:39.546617 27436 master.cpp:428] Master only allowing authenticated
slaves to register
I0925 19:15:39.546632 27436 credentials.hpp:37] Loading credentials for
authentication from
'/tmp/FetcherCacheTest_LocalUncachedExtract_jHBfeA/credentials'
I0925 19:15:39.546931 27436 master.cpp:467] Using default 'crammd5'
authenticator
I0925 19:15:39.547044 27436 master.cpp:504] Authorization enabled
I0925 19:15:39.547276 27441 whitelist_watcher.cpp:79] No whitelist given
I0925 19:15:39.547320 27434 hierarchical.hpp:468] Initialized hierarchical
allocator process
I0925 19:15:39.547471 27438 replica.cpp:641] Replica in STARTING status
received a broadcasted recover request
I0925 19:15:39.548318 27443 recover.cpp:195] Received a recover response from a
replica in STARTING status
I0925 19:15:39.549067 27435 recover.cpp:566] Updating replica status to VOTING
I0925 19:15:39.549115 27440 master.cpp:1603] The newly elected leader is
master@172.17.1.195:41781 with id c8bf1c95-50f4-4832-a570-c560f0b466ae
I0925 19:15:39.549162 27440 master.cpp:1616] Elected as the leading master!
I0925 19:15:39.549190 27440 master.cpp:1376] Recovering from registrar
I0925 19:15:39.549342 27434 registrar.cpp:309] Recovering registrar
I0925 19:15:39.549666 27430 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 418187ns
I0925 19:15:39.549753 27430 replica.cpp:323] Persisted replica status to VOTING
I0925 19:15:39.550089 27442 recover.cpp:580] Successfully joined the Paxos group
I0925 19:15:39.550320 27442 recover.cpp:464] Recover process terminated
I0925 19:15:39.550904 27430 log.cpp:661] Attempting to start the writer
I0925 19:15:39.551955 27434 replica.cpp:477] Replica received implicit promise
request with proposal 1
I0925 19:15:39.552351 27434 leveldb.cpp:306] Persisting metadata (8 bytes) to
leveldb took 380746ns
I0925 19:15:39.552372 27434 replica.cpp:345] Persisted promised to 1
I
