[jira] [Issue Comment Deleted] (MESOS-2077) Ensure that TASK_LOSTs for a hard slave drain (SIGUSR1) include a Reason.

[Guangya Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangya Liu updated MESOS-2077:
---
Comment: was deleted

(was: A RR is here: https://reviews.apache.org/r/38214)

> Ensure that TASK_LOSTs for a hard slave drain (SIGUSR1) include a Reason.
> -
>
> Key: MESOS-2077
> URL: https://issues.apache.org/jira/browse/MESOS-2077
> Project: Mesos
>  Issue Type: Improvement
>  Components: master, slave
>Reporter: Benjamin Mahler
>Assignee: Guangya Liu
>  Labels: twitter
>
> For maintenance, sometimes operators will force the drain of a slave (via 
> SIGUSR1), when deemed safe (e.g. non-critical tasks running) and/or necessary 
> (e.g. bad hardware).
> To eliminate alerting noise, we'd like to add a 'Reason' that expresses the 
> forced drain of the slave, so that these are not considered to be a generic 
> slave removal TASK_LOST.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-2077) Ensure that TASK_LOSTs for a hard slave drain (SIGUSR1) include a Reason.

[Guangya Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14986888#comment-14986888
 ] 

Guangya Liu commented on MESOS-2077:


RR:
https://reviews.apache.org/r/38214/ PROTO: Add a REASON for TASK_LOST with hard 
slave drain
https://reviews.apache.org/r/39857/ Master/Slave: Add a REASON for TASK_LOST 
with hard slave drain

> Ensure that TASK_LOSTs for a hard slave drain (SIGUSR1) include a Reason.
> -
>
> Key: MESOS-2077
> URL: https://issues.apache.org/jira/browse/MESOS-2077
> Project: Mesos
>  Issue Type: Improvement
>  Components: master, slave
>Reporter: Benjamin Mahler
>Assignee: Guangya Liu
>  Labels: twitter
>
> For maintenance, sometimes operators will force the drain of a slave (via 
> SIGUSR1), when deemed safe (e.g. non-critical tasks running) and/or necessary 
> (e.g. bad hardware).
> To eliminate alerting noise, we'd like to add a 'Reason' that expresses the 
> forced drain of the slave, so that these are not considered to be a generic 
> slave removal TASK_LOST.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3560) JSON-based credential files do not work correctly

[Isabel Jimenez (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987711#comment-14987711
 ] 

Isabel Jimenez commented on MESOS-3560:
---

[~neilconway] I sent an email to the dev mailing list to make a census on who 
this might perturb. There wasn't any response from users being concerned about 
this. I'm posting on RB the changes to upgrades.md and will up the email one 
more time.

> JSON-based credential files do not work correctly
> -
>
> Key: MESOS-3560
> URL: https://issues.apache.org/jira/browse/MESOS-3560
> Project: Mesos
>  Issue Type: Bug
>  Components: master
>Reporter: Michael Park
>Assignee: Isabel Jimenez
>  Labels: mesosphere
> Fix For: 0.26.0
>
>
> Specifying the following credentials file:
> {code}
> {
>   “credentials”: [
> {
>   “principal”: “user”,
>   “secret”: “password”
> }
>   ]
> }
> {code}
> Then hitting a master endpoint with:
> {code}
> curl -i -u “user:password” ...
> {code}
> Does not work. This is contrary to the text-based credentials file which 
> works:
> {code}
> user password
> {code}
> Currently, the password in a JSON-based credentials file needs to be 
> base64-encoded in order for it to work:
> {code}
> {
>   “credentials”: [
> {
>   “principal”: “user”,
>   “secret”: “cGFzc3dvcmQ=”
> }
>   ]
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3735) Mesos master should expose the version of registered agents

[Vinod Kone (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kone updated MESOS-3735:
--
Summary: Mesos master should expose the version of registered agents  (was: 
Mesos master should expose the version of registered agents/schedulers )

I've committed the above 2 patches. But I will split this ticket into 3 tickets 
for easy reference.

1) expose the version of agents <-- r/39421
2) add /frameworks endpoint <-- r/39432
3) expose the version of frameworks <-- still needs a review

> Mesos master should expose the version of registered agents
> ---
>
> Key: MESOS-3735
> URL: https://issues.apache.org/jira/browse/MESOS-3735
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Vinod Kone
>Assignee: Guangya Liu
>
> Currently Mesos master doesn't expose the (release and API) version  
> information of clients (agents and schedulers) registered with it. It would 
> be useful to have this information in the WebUI and in an HTTP endpoint. The 
> latter would be especially useful during deploys/upgrades.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MESOS-3388) Add an interface to allow Slave Modules to checkpoint/restore state.

[Greg Mann (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Mann reassigned MESOS-3388:


Assignee: Greg Mann

> Add an interface to allow Slave Modules to checkpoint/restore state.
> 
>
> Key: MESOS-3388
> URL: https://issues.apache.org/jira/browse/MESOS-3388
> Project: Mesos
>  Issue Type: Bug
>Reporter: Kapil Arya
>Assignee: Greg Mann
>
> * This is to restore module-specific in-memory data structures that might be 
> required by the modules to do cleanup on task exit, etc.
> * We need to define the interaction when an Agent is restarted with a 
> different set of modules.
> One open question is how does an Agent identify a certain module? One 
> possibility is to assign a UID to the module and pass it in during 
> `create()`?. The UID is used to assign a ckpt directory during ckpt/restore. 
> (Something like /tmp/mesos/...//modules/).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987934#comment-14987934
 ] 

Stephen Degler commented on MESOS-3787:
---

The patch by [~haosd...@gmail.com] does not have the desired effect.   Here is 
the output:
root 24227  0.0  0.0   4196   460 ?S19:23   0:00 sh -c docker 
run -c 8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e/${KRB5CCNAME}:${KRB5CCNAME}:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=${KRB5CCNAME} --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 
.. < stuff deleted > ..

root 24229  0.0  0.0  92596 18140 ?Sl   19:23   0:00 docker run -c 
8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e//tmp/Jj3TH1:/tmp/Jj3TH1:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=/tmp/Jj3TH1 --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 ..  ...


> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3824) Add /frameworks endpoint to master

[Vinod Kone (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987801#comment-14987801
 ] 

Vinod Kone commented on MESOS-3824:
---

I've committed the endpoint change. But I'm keeping the ticket open for the 
review to update CHANGELOG. Can you please follow up with that?

{code}
commit 6001cfc41c886f34517b07962b3b4a2678f345d7
Author: Guangya Liu 
Date:   Tue Nov 3 10:14:11 2015 -0800

Add /master/frameworks to master endpoint.

Review: https://reviews.apache.org/r/39432


{code}

> Add /frameworks endpoint to master
> --
>
> Key: MESOS-3824
> URL: https://issues.apache.org/jira/browse/MESOS-3824
> Project: Mesos
>  Issue Type: Task
>Reporter: Vinod Kone
>Assignee: Guangya Liu
>
> Just like /slaves endpoint, it would be great to have a /frameworks endpoint 
> that shows information about the registered frameworks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3388) Add an interface to allow Slave Modules to checkpoint/restore state.

[Greg Mann (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Mann updated MESOS-3388:
-
Shepherd: Kapil Arya

> Add an interface to allow Slave Modules to checkpoint/restore state.
> 
>
> Key: MESOS-3388
> URL: https://issues.apache.org/jira/browse/MESOS-3388
> Project: Mesos
>  Issue Type: Bug
>Reporter: Kapil Arya
>Assignee: Greg Mann
>
> * This is to restore module-specific in-memory data structures that might be 
> required by the modules to do cleanup on task exit, etc.
> * We need to define the interaction when an Agent is restarted with a 
> different set of modules.
> One open question is how does an Agent identify a certain module? One 
> possibility is to assign a UID to the module and pass it in during 
> `create()`?. The UID is used to assign a ckpt directory during ckpt/restore. 
> (Something like /tmp/mesos/...//modules/).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-3823) Mesos master should expose the version of registered frameworks

[Vinod Kone (JIRA)]

Vinod Kone created MESOS-3823:
-

 Summary: Mesos master should expose the version of registered 
frameworks
 Key: MESOS-3823
 URL: https://issues.apache.org/jira/browse/MESOS-3823
 Project: Mesos
  Issue Type: Improvement
Reporter: Vinod Kone
Assignee: Guangya Liu
 Fix For: 0.26.0


Currently Mesos master doesn't expose the (release and API) version  
information of clients (agents and schedulers) registered with it. It would be 
useful to have this information in the WebUI and in an HTTP endpoint. The 
latter would be especially useful during deploys/upgrades.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3823) Mesos master should expose the version of registered frameworks

[Vinod Kone (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kone updated MESOS-3823:
--
Fix Version/s: (was: 0.26.0)

> Mesos master should expose the version of registered frameworks
> ---
>
> Key: MESOS-3823
> URL: https://issues.apache.org/jira/browse/MESOS-3823
> Project: Mesos
>  Issue Type: Improvement
>Reporter: Vinod Kone
>Assignee: Guangya Liu
>
> Currently Mesos master doesn't expose the (release and API) version  
> information of clients (agents and schedulers) registered with it. It would 
> be useful to have this information in the WebUI and in an HTTP endpoint. The 
> latter would be especially useful during deploys/upgrades.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-3824) Add /frameworks endpoint to master

[Vinod Kone (JIRA)]

Vinod Kone created MESOS-3824:
-

 Summary: Add /frameworks endpoint to master
 Key: MESOS-3824
 URL: https://issues.apache.org/jira/browse/MESOS-3824
 Project: Mesos
  Issue Type: Task
Reporter: Vinod Kone
Assignee: Guangya Liu


Just like /slaves endpoint, it would be great to have a /frameworks endpoint 
that shows information about the registered frameworks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3820) Refactor libprocess initialization to allow for test-only reinitialization of the server socket

[Benjamin Mahler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988342#comment-14988342
 ] 

Benjamin Mahler commented on MESOS-3820:


Why can't process::finalize be used / enhanced to support a subsequent call to 
process::initialize()?

> Refactor libprocess initialization to allow for test-only reinitialization of 
> the server socket
> ---
>
> Key: MESOS-3820
> URL: https://issues.apache.org/jira/browse/MESOS-3820
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess, test
>Reporter: Joseph Wu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> *Background*
> Libprocess initialization includes the spawning of a variety of global 
> processes and the creation of the server socket which listens for incoming 
> requests.  Some properties of the server socket are configured via 
> environment variables, such as the IP and port or the SSL configuration.
> In the case of tests, libprocess is initialized once per test binary.  This 
> means that testing different configurations (SSL in particular) is cumbersome 
> as a separate process would be needed for every test case.
> *Proposal*
> # Follow the [example of the SSL 
> library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
>  and allow tests to declare an internal function for re-initializing a 
> portion of libprocess.
> # Move the [existing creation of the server 
> socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
>  into a {{reinitialize_server_socket}} function.
> # Add any necessary cleanup for swapping server sockets.
> # Consider whether any additional locking is required in the 
> {{reinitialize_server_socket}} function.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3708) improve process::subprocess ABORT message

[Benjamin Mahler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Mahler updated MESOS-3708:
---
Shepherd: Benjamin Mahler

> improve process::subprocess ABORT message
> -
>
> Key: MESOS-3708
> URL: https://issues.apache.org/jira/browse/MESOS-3708
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Reporter: James Peach
>Assignee: James Peach
>Priority: Trivial
>
> When the {{exec}} fails in {{process::subprocess}}, you get a {{execvpe 
> failed ...}} abort message with no indication of what failed to exec. Let's 
> print the path in the message to give a hint to the operator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-2353) Improve performance of the master's state.json endpoint for large clusters.

[Vinod Kone (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988425#comment-14988425
 ] 

Vinod Kone commented on MESOS-2353:
---

Is someone actively working on this? I remember there was a design document at 
some point to split the endpoint.

If no one is actively working, I would like to take it over and implement the 
caching approach.

> Improve performance of the master's state.json endpoint for large clusters.
> ---
>
> Key: MESOS-2353
> URL: https://issues.apache.org/jira/browse/MESOS-2353
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Benjamin Mahler
>  Labels: newbie, scalability, twitter
>
> The master's state.json endpoint consistently takes a long time to compute 
> the JSON result, for large clusters:
> {noformat}
> $ time curl -s -o /dev/null localhost:5050/master/state.json
> Mon Jan 26 22:38:50 UTC 2015
> real  0m13.174s
> user  0m0.003s
> sys   0m0.022s
> {noformat}
> This can cause the master to get backlogged if there are many state.json 
> requests in flight.
> Looking at {{perf}} data, it seems most of the time is spent doing memory 
> allocation / de-allocation. This ticket will try to capture any low hanging 
> fruit to speed this up. Possibly we can leverage moves if they are not 
> already being used by the compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3824) Add /frameworks endpoint to master

[Guangya Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988431#comment-14988431
 ] 

Guangya Liu commented on MESOS-3824:


CHANGELOG RR: https://reviews.apache.org/r/39922/

> Add /frameworks endpoint to master
> --
>
> Key: MESOS-3824
> URL: https://issues.apache.org/jira/browse/MESOS-3824
> Project: Mesos
>  Issue Type: Task
>Reporter: Vinod Kone
>Assignee: Guangya Liu
>
> Just like /slaves endpoint, it would be great to have a /frameworks endpoint 
> that shows information about the registered frameworks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3752) CentOS 6 dependency install fails at Maven

[Greg Mann (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988064#comment-14988064
 ] 

Greg Mann commented on MESOS-3752:
--

Build seems to be working OK now with the updated Maven package, so I'm 
resolving the ticket.

> CentOS 6 dependency install fails at Maven
> --
>
> Key: MESOS-3752
> URL: https://issues.apache.org/jira/browse/MESOS-3752
> Project: Mesos
>  Issue Type: Documentation
>Reporter: Greg Mann
>Assignee: Greg Mann
>  Labels: documentation, installation, mesosphere
>
> It seems the Apache Maven dependencies have changed such that following the 
> Getting Started docs for CentOS 6.6 will fail at Maven installation:
> {code}
> ---> Package apache-maven.noarch 0:3.3.3-2.el6 will be installed
> --> Processing Dependency: java-devel >= 1:1.7.0 for package: 
> apache-maven-3.3.3-2.el6.noarch
> --> Finished Dependency Resolution
> Error: Package: apache-maven-3.3.3-2.el6.noarch (epel-apache-maven)
>Requires: java-devel >= 1:1.7.0
>Available: java-1.5.0-gcj-devel-1.5.0.0-29.1.el6.x86_64 (base)
>java-devel = 1.5.0
>Available: 
> 1:java-1.6.0-openjdk-devel-1.6.0.35-1.13.7.1.el6_6.x86_64 (base)
>java-devel = 1:1.6.0
>Available: 
> 1:java-1.6.0-openjdk-devel-1.6.0.36-1.13.8.1.el6_7.x86_64 (updates)
>java-devel = 1:1.6.0
>  You could try using --skip-broken to work around the problem
>  You could try running: rpm -Va --nofiles --nodigest
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3405) Add JSON::protobuf for google::protobuf::RepeatedPtrField.

[Kapil Arya (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988044#comment-14988044
 ] 

Kapil Arya commented on MESOS-3405:
---

Is it possible to have specialization such as  {{JSON::Protobuf(const Labels& 
labels)}} and {{JSON::Protobuf(const Parameters& parameters)}} to simply some 
of the stuff in {{src/http.cpp}}? For a bunch of messages, we have to 
explicitly model the fields because we want to render Labels as:
{code}
{ "labels": [ {"key": "k", "value": "v"}, ...] }
{code}

as opposed to

{code}
{ "labels": { "labels" : [ {"key": "k", "value": "v"}, ...] } }
{code}



> Add JSON::protobuf for google::protobuf::RepeatedPtrField.
> --
>
> Key: MESOS-3405
> URL: https://issues.apache.org/jira/browse/MESOS-3405
> Project: Mesos
>  Issue Type: Task
>  Components: stout
>Reporter: Michael Park
>Assignee: Klaus Ma
>
> Currently, {{stout/protobuf.hpp}} provides a {{JSON::Protobuf}} utility which 
> converts a {{google::protobuf::Message}} into a {{JSON::Object}}.
> We should add the support for {{google::protobuf::RepeatedPtrField}} by 
> introducing overloaded functions.
> {code}
> namespace JSON {
>   Object protobuf(const google::protobuf::Message& message)
>   {
> Object object;
> /* Move the body of JSON::Protobuf constructor here. */
> return object;
>   }
>   template 
>   Array protobuf(const google::protobuf::RepeatedPtrField& repeated)
>   {
> static_assert(std::is_convertible::value,
>   "T must be a google::protobuf::Message");
> JSON::Array array;
> array.values.reserve(repeated.size());
> foreach (const T& elem, repeated) {
>   array.values.push_back(JSON::Protobuf(elem));
> }
> return array;
>   }
> }
> {code}
> The new {{RepeatedPtrField}} version can be used in at least the following 
> places:
> * {{src/common/http.cpp}}
> * {{src/master/http.cpp}}
> * {{src/slave/containerizer/mesos/containerizer.cpp}}
> * {{src/tests/reservation_endpoints_tests.cpp}}
> * {{src/tests/resources_tests.cpp}}: {{ResourcesTest.ParsingFromJSON}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987934#comment-14987934
 ] 

Stephen Degler edited comment on MESOS-3787 at 11/3/15 8:45 PM:


The patch by [~haosd...@gmail.com] does not have the desired effect.   Here is 
the output of ps on the slave:

root 24227  0.0  0.0   4196   460 ?S19:23   0:00 sh -c docker 
run -c 8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e/${KRB5CCNAME}:${KRB5CCNAME}:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=${KRB5CCNAME} --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 
.. < stuff deleted > ..

root 24229  0.0  0.0  92596 18140 ?Sl   19:23   0:00 docker run -c 
8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e//tmp/Jj3TH1:/tmp/Jj3TH1:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=/tmp/Jj3TH1 --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 ..  ...



was (Author: sdegler):
The patch by [~haosd...@gmail.com] does not have the desired effect.   Here is 
the output:
root 24227  0.0  0.0   4196   460 ?S19:23   0:00 sh -c docker 
run -c 8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e/${KRB5CCNAME}:${KRB5CCNAME}:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=${KRB5CCNAME} --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 
.. < stuff deleted > ..

root 24229  0.0  0.0  92596 18140 ?Sl   19:23   0:00 docker run -c 
8192 -m 34359738368 -e MARATHON_APP_VERSION=2015-11-03T17:02:56.484Z -e 
HOST=slave-net1-1.simfarm2.cnje1.twosigma.com -e PORT_1=31938 -e 
MESOS_TASK_ID=test-13.6d001069-8260-11e5-830c-fa163ecc88fa -e PORT=31938 -e 
PORTS=31938 -e MARATHON_APP_ID=/test-13 -e PORT0=31938 -e 
MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/sss:/var/lib/sss:rw -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e//tmp/Jj3TH1:/tmp/Jj3TH1:ro
 -v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e:/mnt/mesos/sandbox
 --net host --env=KRB5CCNAME=/tmp/Jj3TH1 --user=10138:5000 --name 
mesos-20151103-163728-312349868-5050-3623-S0.d7bb20ed-970d-4aab-bc84-ddf76361741e
 ..  ...


> As a developer, I'd 

[jira] [Commented] (MESOS-3814) Add checks to make sure isolators and the launcher are compatible.

[Jie Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988627#comment-14988627
 ] 

Jie Yu commented on MESOS-3814:
---

A stop gap fix has been committed first.

commit 082d10e5feefd72ed300e593019ec56e18418142
Author: Artem Harutyunyan 
Date:   Tue Nov 3 16:09:09 2015 -0800

Used 'cgroups::enabled' to check for the availability of the freezer.

Review: https://reviews.apache.org/r/39841

> Add checks to make sure isolators and the launcher are compatible.
> --
>
> Key: MESOS-3814
> URL: https://issues.apache.org/jira/browse/MESOS-3814
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 0.26.0
>Reporter: Jie Yu
>Assignee: Artem Harutyunyan
>  Labels: mesosphere, twitter
>
> There's a recent change regarding the picking of which launcher (Linux or 
> Posix) to use
> https://reviews.apache.org/r/39604
> In our environment, cgroups are not auto-mounted after reboot. We rely on 
> Mesos itself to mount all relevant cgroups hierachies.
> After the reboot, the above patch detects that 'freezer' hierarchy is not 
> mounted, therefore, decided to use the Posix launcher (if --launcher is not 
> specified explictly).
> Port mapping isolator requires network namespace to be created for each 
> container (thus requires Linux launcher). But we don't have a check to verify 
> that launcher and isolators are compatible.
> Slave thus starts fine and task failed with weird error like:
> {noformat}
> Collect failed: Failed to create the ingress qdisc on mesos61099: Link 
> 'mesos61099' is not found.
> {noformat}
> It does take us quite a few time to figure out the root cause.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-2353) Improve performance of the master's state.json endpoint for large clusters.

[Michael Park (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988659#comment-14988659
 ] 

Michael Park commented on MESOS-2353:
-

[~vinodkone] I've been working on a library to avoid all of the unnecessary 
intermediate object construction going from C++ to JSON string representation 
to improve performance. I'll send out a design doc based on experimentation and 
prototyping soon.

> Improve performance of the master's state.json endpoint for large clusters.
> ---
>
> Key: MESOS-2353
> URL: https://issues.apache.org/jira/browse/MESOS-2353
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Benjamin Mahler
>  Labels: newbie, scalability, twitter
>
> The master's state.json endpoint consistently takes a long time to compute 
> the JSON result, for large clusters:
> {noformat}
> $ time curl -s -o /dev/null localhost:5050/master/state.json
> Mon Jan 26 22:38:50 UTC 2015
> real  0m13.174s
> user  0m0.003s
> sys   0m0.022s
> {noformat}
> This can cause the master to get backlogged if there are many state.json 
> requests in flight.
> Looking at {{perf}} data, it seems most of the time is spent doing memory 
> allocation / de-allocation. This ticket will try to capture any low hanging 
> fruit to speed this up. Possibly we can leverage moves if they are not 
> already being used by the compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988733#comment-14988733
 ] 

Stephen Degler commented on MESOS-3787:
---



Also the shell footprint isn't a big deal, however inserting exec after bash -c 
should suffice to get rid of it.
I think problem is that the variables are quoted in some way that bash is not 
expanding them.
The ${KRB5CCNAME} should already be expanded when you see it in ps.
Otherwise, I have no problem with this approach.   

 Original message 
From: "haosdent (JIRA)"  
Date: 11/03/2015  8:39 PM  (GMT-05:00) 
To: step...@degler.net 
Subject: [jira] [Commented] (MESOS-3787) As a developer, I'd like to be able
  to expand environment variables through the Docker executor. 


    [ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988693#comment-14988693
 ] 

haosdent commented on MESOS-3787:
-

Because have two process?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)



> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988747#comment-14988747
 ] 

Stephen Degler commented on MESOS-3787:
---

Yes I see thtat the subprocess has them expanded.   Let me double check and see 
what is going on.  I didn't successfully authenticate, but perhaps I missed 
something else.


> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3820) Refactor libprocess initialization to allow for test-only reinitialization of the server socket

[Joseph Wu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988698#comment-14988698
 ] 

Joseph Wu commented on MESOS-3820:
--

I'll investigate that approach.  

It seems like swapping out the server socket does not do the trick ([attempted 
here|https://github.com/kaysoky/mesos/commits/process_reinit]).

We still want to make any re-initialization test-only though.  So most likely, 
I'll need to investigate how to fully {{finalize}} libprocess.  Right now, we 
only clean up the {{process_manager}}.

> Refactor libprocess initialization to allow for test-only reinitialization of 
> the server socket
> ---
>
> Key: MESOS-3820
> URL: https://issues.apache.org/jira/browse/MESOS-3820
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess, test
>Reporter: Joseph Wu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> *Background*
> Libprocess initialization includes the spawning of a variety of global 
> processes and the creation of the server socket which listens for incoming 
> requests.  Some properties of the server socket are configured via 
> environment variables, such as the IP and port or the SSL configuration.
> In the case of tests, libprocess is initialized once per test binary.  This 
> means that testing different configurations (SSL in particular) is cumbersome 
> as a separate process would be needed for every test case.
> *Proposal*
> # Follow the [example of the SSL 
> library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
>  and allow tests to declare an internal function for re-initializing a 
> portion of libprocess.
> # Move the [existing creation of the server 
> socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
>  into a {{reinitialize_server_socket}} function.
> # Add any necessary cleanup for swapping server sockets.
> # Consider whether any additional locking is required in the 
> {{reinitialize_server_socket}} function.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3820) Refactor libprocess initialization to allow for test-only reinitialization of the server socket

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3820:
-
Story Points: 5  (was: 3)

> Refactor libprocess initialization to allow for test-only reinitialization of 
> the server socket
> ---
>
> Key: MESOS-3820
> URL: https://issues.apache.org/jira/browse/MESOS-3820
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess, test
>Reporter: Joseph Wu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> *Background*
> Libprocess initialization includes the spawning of a variety of global 
> processes and the creation of the server socket which listens for incoming 
> requests.  Some properties of the server socket are configured via 
> environment variables, such as the IP and port or the SSL configuration.
> In the case of tests, libprocess is initialized once per test binary.  This 
> means that testing different configurations (SSL in particular) is cumbersome 
> as a separate process would be needed for every test case.
> *Proposal*
> # Follow the [example of the SSL 
> library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
>  and allow tests to declare an internal function for re-initializing a 
> portion of libprocess.
> # Move the [existing creation of the server 
> socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
>  into a {{reinitialize_server_socket}} function.
> # Add any necessary cleanup for swapping server sockets.
> # Consider whether any additional locking is required in the 
> {{reinitialize_server_socket}} function.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[haosdent (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988693#comment-14988693
 ] 

haosdent commented on MESOS-3787:
-

Because have two process?

> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988721#comment-14988721
 ] 

Stephen Degler commented on MESOS-3787:
---



Because should have been expanded by the time is visible in /bin/ps


Sent from my Verizon Wireless 4G LTE smartphone

 Original message 
From: "haosdent (JIRA)"  
Date: 11/03/2015  8:39 PM  (GMT-05:00) 
To: step...@degler.net 
Subject: [jira] [Commented] (MESOS-3787) As a developer, I'd like to be able
  to expand environment variables through the Docker executor. 


    [ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988693#comment-14988693
 ] 

haosdent commented on MESOS-3787:
-

Because have two process?




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)



> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-1563) Failed to configure on FreeBSD

[David Forsythe (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988673#comment-14988673
 ] 

David Forsythe commented on MESOS-1563:
---

[~idownes] would it be appropriate to have this ticket assigned to me?

Also, what is the etiquette for filing tickets against unsupported platforms? 
I'd like to start creating issues for things that need to be done after this 
change set lands.

> Failed to configure on FreeBSD
> --
>
> Key: MESOS-1563
> URL: https://issues.apache.org/jira/browse/MESOS-1563
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 0.19.0
> Environment: FreeBSD-10/stable
>Reporter: Dmitry Sivachenko
>
> When trying to configure mesos on FreeBSD, I get the following error:
> configure: Setting up build environment for x86_64 freebsd10.0
> configure: error: "Mesos is currently unsupported on your platform."
> Why? Is there anything really Linux-specific inside? It's written in Java 
> after all.
> And MacOS is supported, but it is rather close to FreeBSD.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3820) Refactor libprocess initialization to allow for test-only reinitialization of the server socket

[Joseph Wu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Wu updated MESOS-3820:
-
Description: 
*Background*
Libprocess initialization includes the spawning of a variety of global 
processes and the creation of the server socket which listens for incoming 
requests.  Some properties of the server socket are configured via environment 
variables, such as the IP and port or the SSL configuration.

In the case of tests, libprocess is initialized once per test binary.  This 
means that testing different configurations (SSL in particular) is cumbersome 
as a separate process would be needed for every test case.

*Proposal* (Still under investigation)
# Augment {{process::finalize}} to completely clean up libprocess.
#* {{process_manager}}
#* {{socket_manager}}
#* {{EventLoop}}
#* {{Clock}}
#* {{__s__}}
#* {{__address__}}
#* Garbage collector, help, logging, profiler, statistics, route processes 
(should fall under {{process_manager}}).
#* {{mime}}
# Add a test-only {{process::reinitialize}} function, which should be roughly 
equivalent to a first-time run of {{process::initialize}}.

-*Proposal to swap out server socket*- (Does not work)
# Follow the [example of the SSL 
library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
 and allow tests to declare an internal function for re-initializing a portion 
of libprocess.
# Move the [existing creation of the server 
socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
 into a {{reinitialize_server_socket}} function.
# Add any necessary cleanup for swapping server sockets.
# Consider whether any additional locking is required in the 
{{reinitialize_server_socket}} function.

  was:
*Background*
Libprocess initialization includes the spawning of a variety of global 
processes and the creation of the server socket which listens for incoming 
requests.  Some properties of the server socket are configured via environment 
variables, such as the IP and port or the SSL configuration.

In the case of tests, libprocess is initialized once per test binary.  This 
means that testing different configurations (SSL in particular) is cumbersome 
as a separate process would be needed for every test case.

*Proposal*
# Follow the [example of the SSL 
library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
 and allow tests to declare an internal function for re-initializing a portion 
of libprocess.
# Move the [existing creation of the server 
socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
 into a {{reinitialize_server_socket}} function.
# Add any necessary cleanup for swapping server sockets.
# Consider whether any additional locking is required in the 
{{reinitialize_server_socket}} function.


> Refactor libprocess initialization to allow for test-only reinitialization of 
> the server socket
> ---
>
> Key: MESOS-3820
> URL: https://issues.apache.org/jira/browse/MESOS-3820
> Project: Mesos
>  Issue Type: Task
>  Components: libprocess, test
>Reporter: Joseph Wu
>Assignee: Joseph Wu
>  Labels: mesosphere
>
> *Background*
> Libprocess initialization includes the spawning of a variety of global 
> processes and the creation of the server socket which listens for incoming 
> requests.  Some properties of the server socket are configured via 
> environment variables, such as the IP and port or the SSL configuration.
> In the case of tests, libprocess is initialized once per test binary.  This 
> means that testing different configurations (SSL in particular) is cumbersome 
> as a separate process would be needed for every test case.
> *Proposal* (Still under investigation)
> # Augment {{process::finalize}} to completely clean up libprocess.
> #* {{process_manager}}
> #* {{socket_manager}}
> #* {{EventLoop}}
> #* {{Clock}}
> #* {{__s__}}
> #* {{__address__}}
> #* Garbage collector, help, logging, profiler, statistics, route processes 
> (should fall under {{process_manager}}).
> #* {{mime}}
> # Add a test-only {{process::reinitialize}} function, which should be roughly 
> equivalent to a first-time run of {{process::initialize}}.
> -*Proposal to swap out server socket*- (Does not work)
> # Follow the [example of the SSL 
> library|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/openssl.cpp#L280]
>  and allow tests to declare an internal function for re-initializing a 
> portion of libprocess.
> # Move the [existing creation of the server 
> socket|https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/process.cpp#L852-L856]
>  into a {{reinitialize_server_socket}} function.
> # Add any 

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[haosdent (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988726#comment-14988726
 ] 

haosdent commented on MESOS-3787:
-

I notice the second item in you post about "ps" have already show the variables.

> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988809#comment-14988809
 ] 

Stephen Degler commented on MESOS-3787:
---

Ah I see:
Something like this:"volumes": [
{
"containerPath": "${KRB5CCNAME}",
"hostPath": "${KRB5CCNAME}",
"mode": "RO"
}
  ]

was transformed differently because it was treated as if it had a relative path?

-v 
/home/tsram/mesos-slave/work/slaves/20151103-163728-312349868-5050-3623-S0/frameworks/20151103-163728-312349868-5050-3623-/executors/test-13.6d001069-8260-11e5-830c-fa163ecc88fa/runs/d7bb20ed-970d-4aab-bc84-ddf76361741e//tmp/Jj3TH1:/tmp/Jj3TH1:ro

This is due to something about handling relative vs absolute paths.

-v /tmp/Jj3TH1:/tmp/Jj3TH1:ro

Would probably work just fine.

So your patch is good but then the logic with handling relative paths is 
mangled by the presence of the variables.


> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3602) hdfs du fails due to prepended / on path

[haosdent (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988800#comment-14988800
 ] 

haosdent commented on MESOS-3602:
-

I think you could add [~bernd-mesos] as reviewer :-) . He is the author of 
fetcher. 

> hdfs du fails due to prepended / on path
> 
>
> Key: MESOS-3602
> URL: https://issues.apache.org/jira/browse/MESOS-3602
> Project: Mesos
>  Issue Type: Bug
>  Components: fetcher
>Affects Versions: 0.23.0
>Reporter: alexius ludeman
>Assignee: James Peach
>
> hdfs.hpp du() fails to run.  It appears to prepend "/" but the path passed in 
> is a uri of something like "hdfs:///a/path/to/artifact.tar.gz".
> {code}
> W1007 13:46:25.791894 373116928 fetcher.cpp:436] Reverting to fetching 
> directly into the sandbox for 
> 'hdfs:///a/path/to/3.3.2-SNAPSHOT/executor-3.3.2-SNAPSHOT-artifact-with-dependencies-archive.tar.gz',
>  due to failure to fetch through the cache, with error: Could not determine 
> size of cache file for 
> 'lexinator@hdfs:///a/path/to/3.3.2-SNAPSHOT/executor-3.3.2-SNAPSHOT-artifact-with-dependencies-archive.tar.gz'
>  with error: Hadoop client could not determine size: HDFS du returned an 
> unexpected number of results: '2015-10-07 13:46:21,958 WARN  [main] 
> util.NativeCodeLoader (NativeCodeLoader.java:(62)) - Unable to load 
> native-hadoop library for your platform... using builtin-java classes where 
> applicable
> -du: java.net.URISyntaxException: Expected scheme-specific part at index 5: 
> hdfs:
> Usage: hadoop fs [generic options] -du [-s] [-h]  ...
> {code}
> The command it's running is:
> {code}
> /usr/bin/env bash /.../hadoop fs -du -h 
> /hdfs:///a/path/to/3.3.2-SNAPSHOT/executor-3.3.2-SNAPSHOT-artifact-with-dependencies-archive.tar.gz
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[haosdent (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988814#comment-14988814
 ] 

haosdent commented on MESOS-3787:
-

yes, now if the container path don't have a "/" prefix, would treat it as a 
relative path.

> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MESOS-3787) As a developer, I'd like to be able to expand environment variables through the Docker executor.

[Stephen Degler (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988821#comment-14988821
 ] 

Stephen Degler commented on MESOS-3787:
---

My horrible hack :-)

"volumes": [
{

"containerPath": "/${KRB5CCNAME}",
"hostPath": "/${KRB5CCNAME}",
"mode": "RO"
}
]

So the nice thing about our patch was that substitutions were done earlier.  
its wrong because we probably leak memory as we didn't want to change the 
mutability of taskInfo.  Your patch is much smaller, but then the subsequent 
logic examines the arguments prior to expansion and goofs up the paths at least.




> As a developer, I'd like to be able to expand environment variables through 
> the Docker executor.
> 
>
> Key: MESOS-3787
> URL: https://issues.apache.org/jira/browse/MESOS-3787
> Project: Mesos
>  Issue Type: Wish
>Reporter: John Garcia
>  Labels: mesosphere
> Attachments: mesos.patch, test-example.json
>
>
> We'd like to have expanded variables usable in [the json files used to create 
> a Marathon app, hence] the Task's CommandInfo, so that the executor is able 
> to detect the correct values at runtime.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-3826) Add an optional unique identifier for resource reservations

[Sargun Dhillon (JIRA)]

Sargun Dhillon created MESOS-3826:
-

 Summary: Add an optional unique identifier for resource 
reservations
 Key: MESOS-3826
 URL: https://issues.apache.org/jira/browse/MESOS-3826
 Project: Mesos
  Issue Type: Improvement
  Components: general
Reporter: Sargun Dhillon
Priority: Minor


Thanks to the resource reservation primitives, frameworks can reserve 
resources. These reservations are per role, which means multiple frameworks can 
share reservations. This can get very hairy, as multiple reservations can occur 
on each agent. 

It would be nice to be able to optionally, uniquely identify reservations by 
ID, much like persistent volumes are today. This could be done by adding a new 
protobuf field, such as Resource.ReservationInfo.id, that if set upon 
reservation time, would come back when the reservation is advertised.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (MESOS-3280) Master fails to access replicated log after network partition

[Neil Conway (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988199#comment-14988199
 ] 

Neil Conway edited comment on MESOS-3280 at 11/3/15 9:47 PM:
-

Fix merged in 82b6112cabc838f9bfa, should be in 0.26


was (Author: neilc):
Merged in 82b6112cabc838f9bfa.

> Master fails to access replicated log after network partition
> -
>
> Key: MESOS-3280
> URL: https://issues.apache.org/jira/browse/MESOS-3280
> Project: Mesos
>  Issue Type: Bug
>  Components: master, replicated log
>Affects Versions: 0.23.0
> Environment: Zookeeper version 3.4.5--1
>Reporter: Bernd Mathiske
>Assignee: Neil Conway
>  Labels: mesosphere
> Fix For: 0.26.0
>
> Attachments: rep-log-race-cond-logs.tar.gz, 
> rep-log-startup-race-test-1.patch
>
>
> In a 5 node cluster with 3 masters and 2 slaves, and ZK on each node, when a 
> network partition is forced, all the masters apparently lose access to their 
> replicated log. The leading master halts. Unknown reasons, but presumably 
> related to replicated log access. The others fail to recover from the 
> replicated log. Unknown reasons. This could have to do with ZK setup, but it 
> might also be a Mesos bug. 
> This was observed in a Chronos test drive scenario described in detail here:
> https://github.com/mesos/chronos/issues/511
> With setup instructions here:
> https://github.com/mesos/chronos/issues/508



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (MESOS-3825) Enable mesos-reviewbot project on jenkins to use SSL

[Jojy Varghese (JIRA)]

Jojy Varghese created MESOS-3825:


 Summary: Enable mesos-reviewbot project on jenkins to use SSL
 Key: MESOS-3825
 URL: https://issues.apache.org/jira/browse/MESOS-3825
 Project: Mesos
  Issue Type: Improvement
 Environment: linux
Reporter: Jojy Varghese
Assignee: Jojy Varghese


Currently mesos-reviewbot project does  not support parameterized 
configuration. This limits the project from building using --enable-ssl (and 
others) configuration arguments for building mesos. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MESOS-3826) Add an optional unique identifier for resource reservations

[Guangya Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Guangya Liu updated MESOS-3826:
---
Shepherd: Michael Park

> Add an optional unique identifier for resource reservations
> ---
>
> Key: MESOS-3826
> URL: https://issues.apache.org/jira/browse/MESOS-3826
> Project: Mesos
>  Issue Type: Improvement
>  Components: general
>Reporter: Sargun Dhillon
>Assignee: Guangya Liu
>Priority: Minor
>  Labels: mesosphere
>
> Thanks to the resource reservation primitives, frameworks can reserve 
> resources. These reservations are per role, which means multiple frameworks 
> can share reservations. This can get very hairy, as multiple reservations can 
> occur on each agent. 
> It would be nice to be able to optionally, uniquely identify reservations by 
> ID, much like persistent volumes are today. This could be done by adding a 
> new protobuf field, such as Resource.ReservationInfo.id, that if set upon 
> reservation time, would come back when the reservation is advertised.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (MESOS-3000) Failing test - NsTest.ROOT_setns

[Chris Lambert (JIRA)]

 [ 
https://issues.apache.org/jira/browse/MESOS-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lambert reassigned MESOS-3000:


Assignee: Chris Lambert

> Failing test - NsTest.ROOT_setns
> 
>
> Key: MESOS-3000
> URL: https://issues.apache.org/jira/browse/MESOS-3000
> Project: Mesos
>  Issue Type: Bug
>  Components: test
>Affects Versions: 0.23.0
>Reporter: Ian Downes
>Assignee: Chris Lambert
>Priority: Blocker
>
> Appears to be the same issue plaguing MESOS-2199
> {noformat}
> [root@hostname build]# MESOS_VERBOSE=1 ./bin/mesos-tests.sh 
> --gtest_filter=NsTest.ROOT_setns
> ...
> [==] Running 1 test from 1 test case.
> [--] Global test environment set-up.
> [--] 1 test from NsTest
> [ RUN  ] NsTest.ROOT_setns
> ABORT: (../../../3rdparty/libprocess/src/subprocess.cpp:163): Failed to 
> os::execvpe in childMain: Permission denied*** Aborted at 1436292540 (unix 
> time) try "date -d @1436292540" if you are using GNU date ***
> PC: @ 0x7f7a1229e625 __GI_raise
> *** SIGABRT (@0xfffe0001) received by PID 1 (TID 0x7f7a19afc820) from PID 
> 1; stack trace: ***
> @ 0x7f7a13421710 (unknown)
> @ 0x7f7a1229e625 __GI_raise
> @ 0x7f7a1229fe05 __GI_abort
> @   0x860ba1 (unknown)
> @   0x860bcf (unknown)
> @ 0x7f7a1826f118 (unknown)
> @ 0x7f7a18274594 (unknown)
> @ 0x7f7a18273b88 (unknown)
> @ 0x7f7a18273098 (unknown)
> @  0x1180720 (unknown)
> @  0x117a5d7 (unknown)
> @ 0x7f7a123548fd clone
> ../../src/tests/ns_tests.cpp:121: Failure
> Failed to wait 15secs for status
> [  FAILED  ] NsTest.ROOT_setns (15004 ms)
> [--] 1 test from NsTest (15004 ms total)
> [--] Global test environment tear-down
> ../../src/tests/environment.cpp:441: Failure
> Failed
> Tests completed with child processes remaining:
> -+- 40531 /home/idownes/workspace/mesos/build/src/.libs/lt-mesos-tests 
> --gtest_filter=NsTest.ROOT_setns
>  \--- 40565 /home/idownes/workspace/mesos/build/src/.libs/lt-mesos-tests 
> --gtest_filter=NsTest.ROOT_setns
> [==] 1 test from 1 test case ran. (15034 ms total)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 1 test, listed below:
> [  FAILED  ] NsTest.ROOT_setns
> {noformat}
> Relevant strace for the forked child:
> {noformat}
> ...
> getpid()= 1
> dup2(6, 0) = 0
> dup2(7, 1) = 1
> dup2(8, 2) = 2
> close(6) = 0
> close(7) = 0
> close(8) = 0
> execve("/home/idownes/workspace/mesos/build/src/setns-test-helper", 
> ["setns-test-helper", "SetnsTestHelper"], [/* 24 vars */]) = -1 EACCES 
> (Permission denied)
> write(2, "ABORT: (../../../3rdparty/libpro"..., 62) = 62
> write(2, "Failed to os::execvpe in childMa"..., 53) = 53
> ...
> {noformat}
> Binary that it's trying to exec:
> {noformat}
> [root@hostname build]# stat 
> /home/idownes/workspace/mesos/build/src/setns-test-helper
>   File: `/home/idownes/workspace/mesos/build/src/setns-test-helper'
>   Size: 7948Blocks: 16 IO Block: 4096   regular file
> Device: 801h/2049d  Inode: 22949249Links: 1
> Access: (0755/-rwxr-xr-x)  Uid: (13118/ idownes)   Gid: ( 1500/employee)
> Access: 2015-07-07 17:58:09.569861237 +
> Modify: 2015-07-07 17:58:09.573861290 +
> Change: 2015-07-07 17:58:09.573861290 +
> [root@hostname build]# 
> /home/idownes/workspace/mesos/build/src/setns-test-helper
> Usage: /home/idownes/workspace/mesos/build/src/.libs/lt-setns-test-helper 
>  [OPTIONS]
> Available subcommands:
> help
> SetnsTestHelper
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)