mesos git commit: Printed out the received stream ID in scheduler API.

[vinodkone]

Repository: mesos
Updated Branches:
  refs/heads/master cb0f4a659 -> ba0e582b2


Printed out the received stream ID in scheduler API.

This patch prints out the received stream ID in error output
caused by Mesos-Stream-Id mismatch in calls to scheduler HTTP API.
Expected stream ID is not printed in error output as it may cause
security leak.

Review: https://reviews.apache.org/r/51342/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/ba0e582b
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/ba0e582b
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/ba0e582b

Branch: refs/heads/master
Commit: ba0e582b2bd1a5149b7178e15f652ea33fae90bb
Parents: cb0f4a6
Author: Abhishek Dasgupta 
Authored: Tue Aug 23 21:35:47 2016 -0700
Committer: Vinod Kone 
Committed: Tue Aug 23 21:35:47 2016 -0700

--
 src/master/http.cpp | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/ba0e582b/src/master/http.cpp
--
diff --git a/src/master/http.cpp b/src/master/http.cpp
index 52dd80b..c6bdad6 100644
--- a/src/master/http.cpp
+++ b/src/master/http.cpp
@@ -839,11 +839,11 @@ Future Master::Http::scheduler(
 "All non-subscribe calls should include the 'Mesos-Stream-Id' header");
   }
 
-  if (request.headers.at("Mesos-Stream-Id") !=
-  framework->http.get().streamId.toString()) {
+  const string& streamId = request.headers.at("Mesos-Stream-Id");
+  if (streamId != framework->http.get().streamId.toString()) {
 return BadRequest(
-"The stream ID included in this request didn't match the stream ID "
-"currently associated with framework ID "
+"The stream ID '" + streamId + "' included in this request "
+"didn't match the stream ID currently associated with framework ID "
 + framework->id().value());
   }
 

mesos git commit: Updated website for 1.0.1 release.

[vinodkone]

Repository: mesos
Updated Branches:
  refs/heads/master 9460d097b -> cb0f4a659


Updated website for 1.0.1 release.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/cb0f4a65
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/cb0f4a65
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/cb0f4a65

Branch: refs/heads/master
Commit: cb0f4a659412888f07c1567c26a433697c3a9a92
Parents: 9460d09
Author: Vinod Kone 
Authored: Tue Aug 23 19:08:17 2016 -0700
Committer: Vinod Kone 
Committed: Tue Aug 23 19:10:10 2016 -0700

--
 site/data/releases.yml  | 10 -
 .../blog/2016-08-23-mesos-1-0-1-released.md | 47 
 2 files changed, 55 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/cb0f4a65/site/data/releases.yml
--
diff --git a/site/data/releases.yml b/site/data/releases.yml
index 3a23aa2..1e9bb85 100644
--- a/site/data/releases.yml
+++ b/site/data/releases.yml
@@ -3,8 +3,8 @@
 #   `versions` below.
 # * When publishing a new blog post, we need to append an item to `news` below.
 latest_stable:
-  version: 1.0.0
-  jira_version: 12324944
+  version: 1.0.1
+  jira_version: 12335956
 news:
 ## * If the news is used to announce a release version, the item structure is
 ##   - title: (required)
@@ -15,6 +15,10 @@ news:
 ##   - title: (required)
 ## date: (required)
 ## blog: (required)
+  - title: Mesos 1.0.1 is released!
+date: August 23, 2016
+blog: mesos-1-0-1-released
+target_version: 1.0.1
   - title: Mesos 1.0.0 is released!
 date: July 27, 2016
 blog: mesos-1-0-0-released
@@ -92,6 +96,8 @@ versions:
 ## after we create the ReleaseNote for 0.28.1, we get the link:
 ## 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311242=12335359.
 ## At here, 12335359 is the jira_version of 0.28.1.
+  - version: 1.0.1
+jira_version: 12335956
   - version: 1.0.0
 jira_version: 12324944
   - version: 0.28.2

http://git-wip-us.apache.org/repos/asf/mesos/blob/cb0f4a65/site/source/blog/2016-08-23-mesos-1-0-1-released.md
--
diff --git a/site/source/blog/2016-08-23-mesos-1-0-1-released.md 
b/site/source/blog/2016-08-23-mesos-1-0-1-released.md
new file mode 100644
index 000..ac6561a
--- /dev/null
+++ b/site/source/blog/2016-08-23-mesos-1-0-1-released.md
@@ -0,0 +1,47 @@
+---
+layout: post
+title: Apache Mesos 1.0.1 Released
+permalink: /blog/mesos-1-0-1-released/
+published: true
+post_author:
+  display_name: Vinod Kone
+  twitter: vinodkone
+tags: Release
+---
+
+The latest Mesos release, 1.0.1, is now available for 
[download](http://mesos.apache.org/downloads). This release includes some 
important bug fixes and improvements on top of 1.0.0. It is highly recommended 
to use this version if you are considering using Mesos 1.0. More specifically, 
this release includes the following fixes and improvements:
+
+[MESOS-5388]( https://issues.apache.org/jira/browse/MESOS-5388) - 
MesosContainerizerLaunch flags execute arbitrary commands via shell.
+[MESOS-5862]( https://issues.apache.org/jira/browse/MESOS-5862) - External 
links to .md files broken.
+[MESOS-5911]( https://issues.apache.org/jira/browse/MESOS-5911) - Webui 
redirection to leader in browser does not work
+[MESOS-5913]( https://issues.apache.org/jira/browse/MESOS-5913) - Stale socket 
FD usage when using libevent + SSL.
+[MESOS-5922]( https://issues.apache.org/jira/browse/MESOS-5922) - mesos-agent 
--help exit status is 1
+[MESOS-5923]( https://issues.apache.org/jira/browse/MESOS-5923) - Ubuntu 14.04 
LTS GPU Isolator "/run" directory is noexec
+[MESOS-5927]( https://issues.apache.org/jira/browse/MESOS-5927) - Unable to 
run "scratch" Dockerfiles with Unified Containerizer.
+[MESOS-5928]( https://issues.apache.org/jira/browse/MESOS-5928) - Agent's 
'--version' flag doesn't work
+[MESOS-5930]( https://issues.apache.org/jira/browse/MESOS-5930) - Orphan tasks 
can show up as running after they have finished.
+[MESOS-5943]( https://issues.apache.org/jira/browse/MESOS-5943) - Incremental 
http parsing of URLs leads to decoder error
+[MESOS-5945]( https://issues.apache.org/jira/browse/MESOS-5945) - 
NvidiaVolume::create() should check for root before creating volume
+[MESOS-5959]( https://issues.apache.org/jira/browse/MESOS-5959) - All non-root 
tests fail on GPU machine
+[MESOS-5969]( https://issues.apache.org/jira/browse/MESOS-5969) - Linux 
'MountInfoTable' entries not sorted as expected
+[MESOS-5982]( https://issues.apache.org/jira/browse/MESOS-5982) - NvidiaVolume 
errors out if any binary is missing

[1/2] mesos git commit: Removed a hard CHECK for ExecutorInfo.framework_id.

[bmahler]

Repository: mesos
Updated Branches:
  refs/heads/master 3ed895e1e -> 9460d097b


Removed a hard CHECK for ExecutorInfo.framework_id.

Review: https://reviews.apache.org/r/51319


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/9460d097
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/9460d097
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/9460d097

Branch: refs/heads/master
Commit: 9460d097b7479c265577ebb75e0dc94dc8f2228d
Parents: cbdaa07
Author: Benjamin Mahler 
Authored: Mon Aug 22 21:41:34 2016 -0700
Committer: Benjamin Mahler 
Committed: Tue Aug 23 18:51:49 2016 -0700

--
 src/master/validation.cpp |  8 +++--
 src/tests/master_validation_tests.cpp | 58 +-
 2 files changed, 62 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/9460d097/src/master/validation.cpp
--
diff --git a/src/master/validation.cpp b/src/master/validation.cpp
index 3e26c32..3d2965e 100644
--- a/src/master/validation.cpp
+++ b/src/master/validation.cpp
@@ -603,9 +603,11 @@ Option validateFrameworkID(
 {
   CHECK_NOTNULL(framework);
 
-  // Master ensures `ExecutorInfo.framework_id`
-  // is set before calling this method.
-  CHECK(executor.has_framework_id());
+  // The master fills in `ExecutorInfo.framework_id` for
+  // executors used in Launch operations.
+  if (!executor.has_framework_id()) {
+return Error("'ExecutorInfo.framework_id' must be set");
+  }
 
   if (executor.framework_id() != framework->id()) {
 return Error(

http://git-wip-us.apache.org/repos/asf/mesos/blob/9460d097/src/tests/master_validation_tests.cpp
--
diff --git a/src/tests/master_validation_tests.cpp 
b/src/tests/master_validation_tests.cpp
index e1a5030..86b4b22 100644
--- a/src/tests/master_validation_tests.cpp
+++ b/src/tests/master_validation_tests.cpp
@@ -617,7 +617,7 @@ TEST_F(DestroyOperationValidationTest, 
UnknownPersistentVolume)
 class TaskValidationTest : public MesosTest {};
 
 
-TEST_F(TaskValidationTest, TaskUsesInvalidFrameworkID)
+TEST_F(TaskValidationTest, ExecutorUsesInvalidFrameworkID)
 {
   Try master = StartMaster();
   ASSERT_SOME(master);
@@ -657,6 +657,62 @@ TEST_F(TaskValidationTest, TaskUsesInvalidFrameworkID)
 }
 
 
+// The master should fill in the `ExecutorInfo.framework_id`
+// if it is not set by the framework.
+TEST_F(TaskValidationTest, ExecutorMissingFrameworkID)
+{
+  Try master = StartMaster();
+  ASSERT_SOME(master);
+
+  // Start the first slave.
+  MockExecutor exec(DEFAULT_EXECUTOR_ID);
+  TestContainerizer containerizer();
+
+  Owned detector = master.get()->createDetector();
+  Try slave = StartSlave(detector.get(), 
);
+  ASSERT_SOME(slave);
+
+  MockScheduler sched;
+  MesosSchedulerDriver driver(
+  , DEFAULT_FRAMEWORK_INFO, master.get()->pid, DEFAULT_CREDENTIAL);
+
+  EXPECT_CALL(sched, registered(, _, _));
+
+  // Create an executor with a missing framework id.
+  ExecutorInfo executor;
+  executor = DEFAULT_EXECUTOR_INFO;
+  executor.clear_framework_id();
+
+  EXPECT_CALL(sched, resourceOffers(, _))
+.WillOnce(LaunchTasks(executor, 1, 1, 16, "*"))
+.WillRepeatedly(Return()); // Ignore subsequent offers.
+
+  EXPECT_CALL(exec, registered(_, _, _, _))
+.Times(1);
+
+  EXPECT_CALL(exec, launchTask(_, _))
+.WillOnce(SendStatusUpdateFromTask(TASK_RUNNING));
+
+  Future status;
+  EXPECT_CALL(sched, statusUpdate(, _))
+.WillOnce(FutureArg<1>());
+
+  driver.start();
+
+  // The task should pass validation since the framework id
+  // is filled in, and when it reaches the dummy executor
+  // it will fail because the executor just exits.
+  AWAIT_READY(status);
+  EXPECT_EQ(TASK_RUNNING, status->state());
+
+  EXPECT_CALL(exec, shutdown(_))
+.Times(AtMost(1));
+
+  driver.stop();
+  driver.join();
+}
+
+
 TEST_F(TaskValidationTest, TaskUsesCommandInfoAndExecutorInfo)
 {
   Try master = StartMaster();

[2/2] mesos git commit: Documented a bug with the use of `pendingTasks` in the master.

[bmahler]

Documented a bug with the use of `pendingTasks` in the master.

The use of `pendingTasks` cannot distinguish between a duplicate
TaskID and a task that has been killed while pending. This means
that if an invalid or unauthorized task is killed while pending,
TASK_KILLED is sent, and once in Master::_accept, we will also
send TASK_ERROR.

Review: https://reviews.apache.org/r/51317


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/cbdaa076
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/cbdaa076
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/cbdaa076

Branch: refs/heads/master
Commit: cbdaa07648cc2571378f2124630dc5533269fe68
Parents: 3ed895e
Author: Benjamin Mahler 
Authored: Mon Aug 22 20:23:25 2016 -0700
Committer: Benjamin Mahler 
Committed: Tue Aug 23 18:51:49 2016 -0700

--
 src/master/master.cpp | 33 -
 1 file changed, 20 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/cbdaa076/src/master/master.cpp
--
diff --git a/src/master/master.cpp b/src/master/master.cpp
index f62be4e..910293a 100644
--- a/src/master/master.cpp
+++ b/src/master/master.cpp
@@ -3357,11 +3357,13 @@ void Master::accept(
 
   // Add to pending tasks.
   //
-  // NOTE: The task ID here hasn't been validated yet, but it
-  // doesn't matter. If the task ID is not valid, the task won't
-  // be launched anyway. If two tasks have the same ID, the second
-  // one will not be put into 'framework->pendingTasks', therefore
-  // will not be launched.
+  // NOTE: If two tasks have the same ID, the second one will
+  // not be put into 'framework->pendingTasks', therefore
+  // will not be launched (and TASK_ERROR will be sent).
+  // Unfortunately, we can't tell the difference between a
+  // duplicate TaskID and getting killed while pending
+  // (removed from the map). So it's possible that we send
+  // a TASK_ERROR after a TASK_KILLED (see _accept())!
   if (!framework->pendingTasks.contains(task.task_id())) {
 framework->pendingTasks[task.task_id()] = task;
   }
@@ -3751,15 +3753,20 @@ void Master::_accept(
   Future authorization = authorizations.front();
   authorizations.pop_front();
 
-  // NOTE: The task will not be in 'pendingTasks' if
-  // 'killTask()' for the task was called before we are here.
-  // No need to launch the task if it's no longer pending.
-  // However, we still need to check the authorization result
-  // and do the validation so that we can send status update
-  // in case the task has duplicated ID.
-  bool pending = framework->pendingTasks.contains(task.task_id());
+  // The task will not be in `pendingTasks` if it has been
+  // killed in the interim. No need to send TASK_KILLED in
+  // this case as it has already been sent. Note however that
+  // we cannot currently distinguish between the task being
+  // killed and the task having a duplicate TaskID within
+  // `pendingTasks`. Therefore we must still validate the task
+  // to ensure we send the TASK_ERROR in the case that it has a
+  // duplicate TaskID.
+  //
+  // TODO(bmahler): We may send TASK_ERROR after a TASK_KILLED
+  // if a task was killed (removed from `pendingTasks`) *and*
+  // the task is invalid or unauthorized here.
 
-  // Remove from pending tasks.
+  bool pending = framework->pendingTasks.contains(task.task_id());
   framework->pendingTasks.erase(task.task_id());
 
   CHECK(!authorization.isDiscarded());

[mesos] Git Push Summary

[vinodkone]

Repository: mesos
Updated Tags:  refs/tags/1.0.1 [created] 3611eb0b7

svn commit: r14925 - in /release/mesos/1.0.1: ./ mesos-1.0.1.tar.gz mesos-1.0.1.tar.gz.asc mesos-1.0.1.tar.gz.md5

[vinodkone]

Author: vinodkone
Date: Wed Aug 24 01:06:47 2016
New Revision: 14925

Log:
Adding mesos-1.0.1.

Added:
release/mesos/1.0.1/
release/mesos/1.0.1/mesos-1.0.1.tar.gz   (with props)
release/mesos/1.0.1/mesos-1.0.1.tar.gz.asc
release/mesos/1.0.1/mesos-1.0.1.tar.gz.md5

Added: release/mesos/1.0.1/mesos-1.0.1.tar.gz
==
Binary file - no diff available.

Propchange: release/mesos/1.0.1/mesos-1.0.1.tar.gz
--
svn:mime-type = application/octet-stream

Added: release/mesos/1.0.1/mesos-1.0.1.tar.gz.asc
==
--- release/mesos/1.0.1/mesos-1.0.1.tar.gz.asc (added)
+++ release/mesos/1.0.1/mesos-1.0.1.tar.gz.asc Wed Aug 24 01:06:47 2016
@@ -0,0 +1,11 @@
+-BEGIN PGP SIGNATURE-
+Version: GnuPG v1
+
+iQEcBAABAgAGBQJXq8bgAAoJEBf21SiiwsdKOtsH/2lndEMRgufihdNu08FtfW5v
+8eTLImBWLOkz4rJFjX7ZUr8OB3FgH9LVxwXZVo9qHNx49eY6LwjlF7ObBx7EOv8S
+8HYATcoVsRez8f7GpOvGsrYoN92wjS3jLtaDRXc/Ad8EBhGijGMMkB53qoA9aoSj
+PF+BP+e3p+5jNoyCD7Q8P2dqSS3FS36F27gfWmwlblqGv91iCDnzot6lGfqy/Jnv
+aj+QZWhaNCItIdxrvaQIiC9aDssClmX5jwXO00h8RXvUpCOtSDMltKlPihysoqXY
+hTzWBGBsdKpzb4YeWx+gVjuGdGuduk60SHV13YGBnq37mbTpi+1zpqt5m/8jAw8=
+=ieI4
+-END PGP SIGNATURE-

Added: release/mesos/1.0.1/mesos-1.0.1.tar.gz.md5
==
--- release/mesos/1.0.1/mesos-1.0.1.tar.gz.md5 (added)
+++ release/mesos/1.0.1/mesos-1.0.1.tar.gz.md5 Wed Aug 24 01:06:47 2016
@@ -0,0 +1 @@
+mesos-1.0.1.tar.gz: 1A AC A0 0A 0F FB 20 F4  B3 1C 2D CC 24 47 C2 C5

mesos git commit: Added Krzysztof Paprocki to contributors.

[vinodkone]

Repository: mesos
Updated Branches:
  refs/heads/master 963c1ff9c -> 3ed895e1e


Added Krzysztof Paprocki to contributors.

Added myself in order to be able to self-assign Jira.

This closes #160


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/3ed895e1
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/3ed895e1
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/3ed895e1

Branch: refs/heads/master
Commit: 3ed895e1e7aade644e0922ab7017f4f6050a7d5c
Parents: 963c1ff
Author: ChrisPaprocki 
Authored: Tue Aug 23 14:55:36 2016 -0700
Committer: Vinod Kone 
Committed: Tue Aug 23 14:55:36 2016 -0700

--
 docs/contributors.yaml | 6 ++
 1 file changed, 6 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/3ed895e1/docs/contributors.yaml
--
diff --git a/docs/contributors.yaml b/docs/contributors.yaml
index 3f06000..d3f4afc 100644
--- a/docs/contributors.yaml
+++ b/docs/contributors.yaml
@@ -289,6 +289,12 @@
   jira_user: klaus1982
   reviewboard_user: klaus1982
 
+- name: Krzysztof Paprocki
+  emails:
+- paprocki.krzysz...@gmail.com
+  jira_user: Kris-Paprocki
+  reviewboard_user: Kris-Paprocki
+
 - name: Lily Chen
   affiliations:
 - {organization: Mesosphere}

[2/2] mesos git commit: Added a CHECK for resource consumption invariant when launching task.

[bmahler]

Added a CHECK for resource consumption invariant when launching task.

Review: https://reviews.apache.org/r/51318


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/963c1ff9
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/963c1ff9
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/963c1ff9

Branch: refs/heads/master
Commit: 963c1ff9cd9443f6a2fa8ab389da7f9287a4c1ef
Parents: 40e1a56
Author: Benjamin Mahler 
Authored: Mon Aug 22 20:30:23 2016 -0700
Committer: Benjamin Mahler 
Committed: Tue Aug 23 14:29:55 2016 -0700

--
 src/master/master.cpp | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/963c1ff9/src/master/master.cpp
--
diff --git a/src/master/master.cpp b/src/master/master.cpp
index d94a851..f62be4e 100644
--- a/src/master/master.cpp
+++ b/src/master/master.cpp
@@ -3840,7 +3840,12 @@ void Master::_accept(
 
   // Add task.
   if (pending) {
-_offeredResources -= addTask(task_, framework, slave);
+const Resources consumed = addTask(task_, framework, slave);
+
+CHECK(_offeredResources.contains(consumed))
+  << _offeredResources << " does not contain " << consumed;
+
+_offeredResources -= consumed;
 
 // TODO(bmahler): Consider updating this log message to
 // indicate when the executor is also being launched.

[1/2] mesos git commit: Documented 'Kill' behavior for task groups.

[bmahler]

Repository: mesos
Updated Branches:
  refs/heads/master dcc8bd7d2 -> 963c1ff9c


Documented 'Kill' behavior for task groups.

Review: https://reviews.apache.org/r/51316


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/40e1a56f
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/40e1a56f
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/40e1a56f

Branch: refs/heads/master
Commit: 40e1a56fc6195a23d91add4b524ff001fe0c73ea
Parents: dcc8bd7
Author: Benjamin Mahler 
Authored: Mon Aug 22 19:48:21 2016 -0700
Committer: Benjamin Mahler 
Committed: Tue Aug 23 14:29:29 2016 -0700

--
 include/mesos/scheduler/scheduler.proto| 7 +++
 include/mesos/v1/scheduler/scheduler.proto | 7 +++
 2 files changed, 14 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/40e1a56f/include/mesos/scheduler/scheduler.proto
--
diff --git a/include/mesos/scheduler/scheduler.proto 
b/include/mesos/scheduler/scheduler.proto
index ac5bce9..0576d90 100644
--- a/include/mesos/scheduler/scheduler.proto
+++ b/include/mesos/scheduler/scheduler.proto
@@ -287,6 +287,13 @@ message Call {
   // receives a terminal update (See TaskState in mesos.proto) for it.
   // If the task is unknown to the master, a TASK_LOST update is
   // generated.
+  //
+  // If a task within a task group is killed before the group is
+  // delivered to the executor, all tasks in the task group are
+  // killed. When a task group has been delivered to the executor,
+  // it is up to the executor to decide how to deal with the kill.
+  // Note The default Mesos executor will currently kill all the
+  // tasks in the task group if it gets a kill for any task.
   message Kill {
 required TaskID task_id = 1;
 optional SlaveID slave_id = 2;

http://git-wip-us.apache.org/repos/asf/mesos/blob/40e1a56f/include/mesos/v1/scheduler/scheduler.proto
--
diff --git a/include/mesos/v1/scheduler/scheduler.proto 
b/include/mesos/v1/scheduler/scheduler.proto
index 05ff0e1..a922a7f 100644
--- a/include/mesos/v1/scheduler/scheduler.proto
+++ b/include/mesos/v1/scheduler/scheduler.proto
@@ -277,6 +277,13 @@ message Call {
   // receives a terminal update (See TaskState in v1/mesos.proto) for
   // it. If the task is unknown to the master, a TASK_LOST update is
   // generated.
+  //
+  // If a task within a task group is killed before the group is
+  // delivered to the executor, all tasks in the task group are
+  // killed. When a task group has been delivered to the executor,
+  // it is up to the executor to decide how to deal with the kill.
+  // Note The default Mesos executor will currently kill all the
+  // tasks in the task group if it gets a kill for any task.
   message Kill {
 required TaskID task_id = 1;
 optional AgentID agent_id = 2;

mesos git commit: Avoided using integers as booleans.

[jieyu]

Repository: mesos
Updated Branches:
  refs/heads/master 3a88c8509 -> dcc8bd7d2


Avoided using integers as booleans.

This is a follow up action for https://reviews.apache.org/r/49352/,
we should avoid using integers as booleans in favor of explicitly
checking the condition we care about.

Review: https://reviews.apache.org/r/50423/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/dcc8bd7d
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/dcc8bd7d
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/dcc8bd7d

Branch: refs/heads/master
Commit: dcc8bd7d2a942889fe473c21ab64e863d0e6a13f
Parents: 3a88c85
Author: Guangya Liu 
Authored: Tue Aug 23 13:55:30 2016 -0700
Committer: Jie Yu 
Committed: Tue Aug 23 13:55:30 2016 -0700

--
 src/docker/docker.cpp   | 2 +-
 src/slave/containerizer/mesos/containerizer.cpp | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/dcc8bd7d/src/docker/docker.cpp
--
diff --git a/src/docker/docker.cpp b/src/docker/docker.cpp
index e07c4aa..0a66046 100755
--- a/src/docker/docker.cpp
+++ b/src/docker/docker.cpp
@@ -470,7 +470,7 @@ Try Docker::Image::create(const 
JSON::Object& json)
   return Error("Unexpected Env format for 'ContainerConfig.Env'");
 }
 
-if (result.count(tokens[0])) {
+if (result.count(tokens[0]) > 0) {
   return Error("Unexpected duplicate environment variables '"
 + tokens[0] + "'");
 }

http://git-wip-us.apache.org/repos/asf/mesos/blob/dcc8bd7d/src/slave/containerizer/mesos/containerizer.cpp
--
diff --git a/src/slave/containerizer/mesos/containerizer.cpp 
b/src/slave/containerizer/mesos/containerizer.cpp
index 1cd3b38..8a8985a 100644
--- a/src/slave/containerizer/mesos/containerizer.cpp
+++ b/src/slave/containerizer/mesos/containerizer.cpp
@@ -1088,7 +1088,7 @@ Future MesosContainerizerProcess::_launch(
 const string& name = variable.name();
 const string& value = variable.value();
 
-if (environment.count(name)) {
+if (environment.count(name) > 0) {
   VLOG(1) << "Overwriting environment variable '"
   << name << "', original: '"
   << environment[name] << "', new: '"
@@ -1145,7 +1145,7 @@ Future MesosContainerizerProcess::_launch(
 const string& name = variable.name();
 const string& value = variable.value();
 
-if (environment.count(name)) {
+if (environment.count(name) > 0) {
   VLOG(1) << "Overwriting environment variable '"
   << name << "', original: '"
   << environment[name] << "', new: '"

mesos git commit: Fixed distcheck build.

[josephwu]

Repository: mesos
Updated Branches:
  refs/heads/master 5b3f34e3c -> 3a88c8509


Fixed distcheck build.

`include/mesos/slave/containerizer.hpp` was removed from the makefile
in: https://reviews.apache.org/r/51263/

Review: https://reviews.apache.org/r/51347/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/3a88c850
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/3a88c850
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/3a88c850

Branch: refs/heads/master
Commit: 3a88c8509378583cf7d6542b1fec1f9d7de3dd19
Parents: 5b3f34e
Author: Joseph Wu 
Authored: Tue Aug 23 13:24:04 2016 -0700
Committer: Joseph Wu 
Committed: Tue Aug 23 13:24:04 2016 -0700

--
 src/Makefile.am | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/3a88c850/src/Makefile.am
--
diff --git a/src/Makefile.am b/src/Makefile.am
index 8dc4175..69e5655 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -602,6 +602,7 @@ agent_HEADERS = 
\
   $(top_srcdir)/include/mesos/agent/agent.hpp  \
   $(top_srcdir)/include/mesos/agent/agent.proto
\
   $(top_srcdir)/include/mesos/slave/container_logger.hpp   \
+  $(top_srcdir)/include/mesos/slave/containerizer.hpp  \
   $(top_srcdir)/include/mesos/slave/isolator.hpp   \
   $(top_srcdir)/include/mesos/slave/oversubscription.hpp   \
   $(top_srcdir)/include/mesos/slave/oversubscription.proto \

mesos git commit: Removed an unnecessary call to stream.precision() in Duration.

[yan]

Repository: mesos
Updated Branches:
  refs/heads/master a064505e4 -> 5b3f34e3c


Removed an unnecessary call to stream.precision() in Duration.


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/5b3f34e3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/5b3f34e3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/5b3f34e3

Branch: refs/heads/master
Commit: 5b3f34e3c6d60d1f8ba4fdc15088b0c9fbd69b14
Parents: a064505
Author: Jiang Yan Xu 
Authored: Tue Aug 23 09:47:19 2016 -0700
Committer: Jiang Yan Xu 
Committed: Tue Aug 23 09:47:19 2016 -0700

--
 3rdparty/stout/include/stout/duration.hpp | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/5b3f34e3/3rdparty/stout/include/stout/duration.hpp
--
diff --git a/3rdparty/stout/include/stout/duration.hpp 
b/3rdparty/stout/include/stout/duration.hpp
index 575a53f..cd4f205 100644
--- a/3rdparty/stout/include/stout/duration.hpp
+++ b/3rdparty/stout/include/stout/duration.hpp
@@ -308,10 +308,8 @@ public:
 
 inline std::ostream& operator<<(std::ostream& stream, const Duration& 
duration_)
 {
-  long precision = stream.precision();
-
-  // Output the duration in full double precision.
-  stream.precision(std::numeric_limits::digits10);
+  // Output the duration in full double precision and save the old precision.
+  long precision = stream.precision(std::numeric_limits::digits10);
 
   // Parse the duration as the sign and the absolute value.
   Duration duration = duration_;

mesos git commit: Added test to simulate slow/unresponsive fetch.

[yan]

Repository: mesos
Updated Branches:
  refs/heads/master 6b1aa0084 -> a064505e4


Added test to simulate slow/unresponsive fetch.

Added test to simulate the scenario of slow/unresponsive HDFS leading
to executor register timeout and verify that slave gets notified of the
failure.

Review: https://reviews.apache.org/r/5/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/a064505e
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/a064505e
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/a064505e

Branch: refs/heads/master
Commit: a064505e411fe78a257e9b336a888f1eeddaa949
Parents: 6b1aa00
Author: Megha Sharma 
Authored: Mon Aug 22 14:51:07 2016 -0700
Committer: Jiang Yan Xu 
Committed: Tue Aug 23 09:42:31 2016 -0700

--
 src/tests/slave_tests.cpp | 129 +
 1 file changed, 129 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/a064505e/src/tests/slave_tests.cpp
--
diff --git a/src/tests/slave_tests.cpp b/src/tests/slave_tests.cpp
index 30ca3da..dcf8454 100644
--- a/src/tests/slave_tests.cpp
+++ b/src/tests/slave_tests.cpp
@@ -263,6 +263,135 @@ TEST_F(SlaveTest, ShutdownUnregisteredExecutor)
 }
 
 
+// This test verifies that mesos agent gets notified of task
+// launch failure triggered by the executor register timeout
+// caused by slow URI fetching.
+TEST_F(SlaveTest, ExecutorTimeoutCausedBySlowFetch)
+{
+  Try master = StartMaster();
+  ASSERT_SOME(master);
+
+  string hadoopPath = os::getcwd();
+  string hadoopBinPath = path::join(hadoopPath, "bin");
+
+  ASSERT_SOME(os::mkdir(hadoopBinPath));
+  ASSERT_SOME(os::chmod(hadoopBinPath, S_IRWXU | S_IRWXG | S_IRWXO));
+
+  // A spurious "hadoop" script that sleeps forever.
+  string mockHadoopScript = "#!/usr/bin/env bash\n"
+"sleep 1000";
+
+  string hadoopCommand = path::join(hadoopBinPath, "hadoop");
+  ASSERT_SOME(os::write(hadoopCommand, mockHadoopScript));
+  ASSERT_SOME(os::chmod(hadoopCommand,
+S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH));
+
+  slave::Flags flags = CreateSlaveFlags();
+  flags.hadoop_home = hadoopPath;
+
+  Fetcher fetcher;
+
+  Try _containerizer = MesosContainerizer::create(
+  flags, true, );
+
+  CHECK_SOME(_containerizer);
+  Owned containerizer(_containerizer.get());
+
+  Owned detector = master.get()->createDetector();
+  Try slave = StartSlave(
+  detector.get(),
+  containerizer.get(),
+  flags);
+  ASSERT_SOME(slave);
+
+  MockScheduler sched;
+  MesosSchedulerDriver driver(
+  ,
+  DEFAULT_FRAMEWORK_INFO,
+  master.get()->pid,
+  DEFAULT_CREDENTIAL);
+
+  EXPECT_CALL(sched, registered(, _, _))
+.Times(1);
+
+  Future offers;
+  EXPECT_CALL(sched, resourceOffers(, _))
+.WillOnce(FutureArg<1>())
+.WillRepeatedly(Return()); // Ignore subsequent offers.
+
+  driver.start();
+
+  AWAIT_READY(offers);
+  EXPECT_NE(0u, offers.get().size());
+
+  // Launch a task with the command executor.
+  // The task uses a URI that needs to be fetched by the HDFS client
+  // and will be blocked until the executor registrartion times out.
+  CommandInfo commandInfo;
+  CommandInfo::URI* uri = commandInfo.add_uris();
+  uri->set_value(path::join("hdfs://dummyhost/dummypath", "test"));
+
+  // Using a dummy command value as it's a required field. The
+  // command won't be invoked.
+  commandInfo.set_value("sleep 10");
+
+  ExecutorID executorId;
+  executorId.set_value("test-executor-staging");
+
+  TaskInfo task = createTask(
+  offers.get()[0].slave_id(),
+  offers.get()[0].resources(),
+  commandInfo,
+  executorId,
+  "test-task-staging");
+
+  Future fetch = FUTURE_DISPATCH(
+  _, ::fetch);
+
+  Future status;
+  EXPECT_CALL(sched, statusUpdate(, _))
+.WillOnce(FutureArg<1>());
+
+  Clock::pause();
+
+  driver.launchTasks(offers.get()[0].id(), {task});
+
+  Future executorLost;
+  EXPECT_CALL(sched, executorLost(, executorId, _, _))
+.WillOnce(FutureSatisfy());
+
+  // Ensure that the slave times out and kills the executor.
+  Future destroyExecutor = FUTURE_DISPATCH(
+  _, ::destroy);
+
+  AWAIT_READY(fetch);
+
+  Clock::advance(flags.executor_registration_timeout);
+
+  AWAIT_READY(destroyExecutor);
+
+  Clock::settle(); // Wait for Containerizer::destroy to complete.
+
+  // Now advance time until the reaper reaps the executor.
+  while (status.isPending()) {
+Clock::advance(process::MAX_REAP_INTERVAL());
+Clock::settle();
+  }
+
+  AWAIT_READY(executorLost);
+
+  AWAIT_READY(status);
+  ASSERT_EQ(TASK_FAILED, status->state());
+  EXPECT_EQ(TaskStatus::SOURCE_SLAVE, 

[1/3] mesos git commit: Fixed broken link in multiple disk documentation.

[tillt]

Repository: mesos
Updated Branches:
  refs/heads/master 6bb9c9bf3 -> 42e8d6fb8


Fixed broken link in multiple disk documentation.

Review: https://reviews.apache.org/r/51283/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/79953e9c
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/79953e9c
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/79953e9c

Branch: refs/heads/master
Commit: 79953e9cd5fcd8fecab3f35c1d77e2e451bbc07a
Parents: 6bb9c9b
Author: Joerg Schad 
Authored: Tue Aug 23 15:17:44 2016 +0200
Committer: Till Toenshoff 
Committed: Tue Aug 23 15:17:44 2016 +0200

--
 docs/multiple-disk.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/79953e9c/docs/multiple-disk.md
--
diff --git a/docs/multiple-disk.md b/docs/multiple-disk.md
index 20b99e0..9bc9f95 100644
--- a/docs/multiple-disk.md
+++ b/docs/multiple-disk.md
@@ -41,7 +41,7 @@ the agent. Data is mapped into the `work_dir` of the agent.
 
 An example resources value for a root disk is shown below. Note that the
 operator could optionally specify a `role` for the disk, which would result in
-[statically reserving](reservations.md) the disk for a single [role](roles.md).
+[statically reserving](reservation.md) the disk for a single [role](roles.md).
 
 {
   "resources" : [
@@ -75,7 +75,7 @@ most 10GB of disk space in total.
 
 An example resources value for a `Path` disk is shown below. Note that the
 operator could optionally specify a `role` for the disk, which would result in
-[statically reserving](reservations.md) the disk for a single [role](roles.md).
+[statically reserving](reservation.md) the disk for a single [role](roles.md).
 
 {
   "resources" : [
@@ -115,7 +115,7 @@ isolation is disabled for `Mount` disks.
 
 An example resources value for a `Mount` disk is shown below. Note that the
 operator could optionally specify a `role` for the disk, which would result in
-[statically reserving](reservations.md) the disk for a single [role](roles.md).
+[statically reserving](reservation.md) the disk for a single [role](roles.md).
 
 {
   "resources" : [

[2/3] mesos git commit: Fixed HTML in upgrades.md.

[tillt]

Fixed HTML in upgrades.md.

Review: https://reviews.apache.org/r/51329/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/158694c2
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/158694c2
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/158694c2

Branch: refs/heads/master
Commit: 158694c241cae4926fa02569df4bd9fe56e89dd5
Parents: 79953e9
Author: Joerg Schad 
Authored: Tue Aug 23 15:17:55 2016 +0200
Committer: Till Toenshoff 
Committed: Tue Aug 23 15:17:55 2016 +0200

--
 docs/upgrades.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/158694c2/docs/upgrades.md
--
diff --git a/docs/upgrades.md b/docs/upgrades.md
index d3e7cdd..a07fb09 100644
--- a/docs/upgrades.md
+++ b/docs/upgrades.md
@@ -180,21 +180,21 @@ We categorize the changes as follows:
 
 ## Upgrading from 0.28.x to 1.0.x ##
 
-
+
 
 * Prior to Mesos 1.0, environment variables prefixed by `SSL_` are used to 
control libprocess SSL support. However, it was found that those environment 
variables may collide with some libraries or programs (e.g., openssl, curl). 
From Mesos 1.0, `SSL_*` environment variables are deprecated in favor of the 
corresponding `LIBPROCESS_SSL_*` variables.
 
-
+
 
 * Prior to Mesos 1.0, Mesos agent recursively changes the ownership of the 
persistent volumes every time they are mounted to a container. From Mesos 1.0, 
this behavior has been changed. Mesos agent will do a _non-recursive_ change of 
ownership of the persistent volumes.
 
-
+
 
 * Mesos 1.0 removed the camel cased protobuf fields in `ContainerConfig` (see 
`include/mesos/slave/isolator.proto`):
   * `required ExecutorInfo executorInfo = 1;`
   * `optional TaskInfo taskInfo = 2;`
 
-
+
 
 * By default, executors will no longer inherit environment variables from the 
agent. The operator can still use the `--executor-environment-variables` flag 
on the agent to explicitly specify what environment variables the executors 
will get. Mesos generated environment variables (i.e., `$MESOS_`, 
`$LIBPROCESS_`) will not be affected. If `$PATH` is not specified for an 
executor, a default value 
`/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin` will be used.
 

[3/3] mesos git commit: Corrected typo in docker_build example.

[tillt]

Corrected typo in docker_build example.

Review: https://reviews.apache.org/r/51147/


Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/42e8d6fb
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/42e8d6fb
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/42e8d6fb

Branch: refs/heads/master
Commit: 42e8d6fb8b894c08b4b0c7cc40036fe630489bba
Parents: 158694c
Author: Joerg Schad 
Authored: Tue Aug 23 15:18:45 2016 +0200
Committer: Till Toenshoff 
Committed: Tue Aug 23 15:18:45 2016 +0200

--
 support/docker_build.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/mesos/blob/42e8d6fb/support/docker_build.sh
--
diff --git a/support/docker_build.sh b/support/docker_build.sh
index 5c38c5c..6ba9cf0 100755
--- a/support/docker_build.sh
+++ b/support/docker_build.sh
@@ -6,7 +6,7 @@ set -xe
 # a given OS and compiler combination.
 
 # Require the following environment variables to be set.
-: ${OS:?"Environment variable 'OS' must be set (e.g., OS=ubuntu14.04)"}
+: ${OS:?"Environment variable 'OS' must be set (e.g., OS=ubuntu:14.04)"}
 : ${BUILDTOOL:?"Environment variable 'BUILDTOOL' must be set (e.g., 
BUILDTOOL=autotools)"}
 : ${COMPILER:?"Environment variable 'COMPILER' must be set (e.g., 
COMPILER=gcc)"}
 : ${CONFIGURATION:?"Environment variable 'CONFIGURATION' must be set (e.g., 
CONFIGURATION='--enable-libevent --enable-ssl')"}