Coverity Scan: Analysis completed for Mesos

[scan-admin]

Your request for analysis of Mesos has been completed successfully.
The results are available at 
https://u2389337.ct.sendgrid.net/wf/click?upn=08onrYu34A-2BWcWUl-2F-2BfV0V05UPxvVjWch-2Bd2MGckcRZ-2B0hUmbDL5L44V5w491gwG_yCAaqzzx-2F-2BA2mRMpk03t3x9hscHw355FKzcsrEtTtpGcYsYiVQewL6zikCpjaQhDLJl645MNfyngLpGyMM3aC2bKqZwhUujdTuGNtGc8ry1AFRmgblQmqEH7pVpFwX7zcdu1qD7I2NFTL8FwlVwucaHnYXfOtkkFWLwZU-2FmJMki-2FUWxxPplTws1dUN-2Fj0Uf0XFSltQRRjhKnxo53xYkznsFUngVyJBO-2B-2BtMdr9qpHxY-3D

Analysis Summary:
   New defects found: 0
   Defects eliminated: 0

Mesos Executor Failing

[Chawla,Sumit]

Hi

I am facing a peculiar issue on one of the slave nodes of our cluster.  I
have a spark cluster with 40+ nodes.  On one of the nodes, all tasks fail
with exit code 0.

ExecutorLostFailure (executor e6745c67-32e8-41ad-b6eb-8fa4d2539da7-S76
exited caused by one of the running tasks) Reason: Unknown executor exit
code (0)


I cannot seem to find anything in mesos-slave.logs, and there is nothing
being written to stdout/stderr.  Are there any debugging utitlities that i
can use to debug what can be getting wrong on that particular slave?

I tried running following but got stuck at:


/mesos-containerizer launch
--command='{"environment":{},"shell":true,"value":"ls -ltr"}'
--directory=/var/tmp/mesos/slaves/e6745c67-32e8-41ad-b6eb-8fa4d2539da7-S77/frameworks/e6745c67-32e8-41ad-b6eb-8fa4d2539da7-0312/executors/e6745c67-32e8-41ad-b6eb-8fa4d2539da7-S77/runs/45aa784c-f485-46a6-aeb8-997e82b80c4f
--help=false --pipe_read=0 --pipe_write=0 --user=smi

Failed to synchronize with slave (it's probably exited)


Would apprecite pointing to any debugging methods/documentation to diagnose
these kind of problems.

Regards
Sumit Chawla

Re: New YouTube channel to house working group recordings?

[James Peach]

> On May 18, 2017, at 4:53 PM, Michael Park  wrote:
> 
> Is there a reason why you want to use new YouTube channel? I think I would
> prefer to use the existing channel and house them in a different playlist.


Is there a link to this from mesos.apache.org? I couldn't find one ...


> On Thu, May 18, 2017 at 5:09 PM Vinod Kone  wrote:
> 
>> +1
>> 
>> @vinodkone
>> 
>>> On May 18, 2017, at 12:29 PM, Judith Malnick 
>> wrote:
>>> 
>>> Hi All,
>>> 
>>> I'd like to start an Apache Mesos YouTube channel to post the recordings
>> of
>>> public meetings and would also like to post the meetings on a new
>> playlist
>>> here , so
>>> that community members in China can access the recordings without VPN.
>>> 
>>> Please let me know your thoughts.
>>> 
>>> Best,
>>> Judith
>>> --
>>> Judith Malnick
>>> DC/OS Community Manager
>>> 310-709-1517
>> 

Re: New YouTube channel to house working group recordings?

[Jie Yu]

Mpark, i think Judith didn't know that there is already a YouTube Channel.
We should just use the one you created.

- Jie

On Thu, May 18, 2017 at 4:53 PM, Michael Park  wrote:

> Is there a reason why you want to use new YouTube channel? I think I would
> prefer to use the existing channel and house them in a different playlist.
>
> On Thu, May 18, 2017 at 5:09 PM Vinod Kone  wrote:
>
> > +1
> >
> > @vinodkone
> >
> > > On May 18, 2017, at 12:29 PM, Judith Malnick 
> > wrote:
> > >
> > > Hi All,
> > >
> > > I'd like to start an Apache Mesos YouTube channel to post the
> recordings
> > of
> > > public meetings and would also like to post the meetings on a new
> > playlist
> > > here ,
> so
> > > that community members in China can access the recordings without VPN.
> > >
> > > Please let me know your thoughts.
> > >
> > > Best,
> > > Judith
> > > --
> > > Judith Malnick
> > > DC/OS Community Manager
> > > 310-709-1517
> >
>

Re: New YouTube channel to house working group recordings?

[Michael Park]

Is there a reason why you want to use new YouTube channel? I think I would
prefer to use the existing channel and house them in a different playlist.

On Thu, May 18, 2017 at 5:09 PM Vinod Kone  wrote:

> +1
>
> @vinodkone
>
> > On May 18, 2017, at 12:29 PM, Judith Malnick 
> wrote:
> >
> > Hi All,
> >
> > I'd like to start an Apache Mesos YouTube channel to post the recordings
> of
> > public meetings and would also like to post the meetings on a new
> playlist
> > here , so
> > that community members in China can access the recordings without VPN.
> >
> > Please let me know your thoughts.
> >
> > Best,
> > Judith
> > --
> > Judith Malnick
> > DC/OS Community Manager
> > 310-709-1517
>

Re: New YouTube channel to house working group recordings?

[Vinod Kone]

+1

@vinodkone

> On May 18, 2017, at 12:29 PM, Judith Malnick  wrote:
> 
> Hi All,
> 
> I'd like to start an Apache Mesos YouTube channel to post the recordings of
> public meetings and would also like to post the meetings on a new playlist
> here , so
> that community members in China can access the recordings without VPN.
> 
> Please let me know your thoughts.
> 
> Best,
> Judith
> -- 
> Judith Malnick
> DC/OS Community Manager
> 310-709-1517

Re: [VOTE] Release Apache Mesos 1.1.2 (rc2)

[Till Toenshoff]

+1

Ran on internal CI. All failures I saw were well documented flaky tests.


> On May 13, 2017, at 12:26 AM, Vinod Kone  wrote:
> 
> +1 (binding)
> 
> Ran on ASF CI. The one configuration that is in "red" is due to a known flaky 
> issue with perf core dump during test suite teardown.
> 
> Revision: 37d98c55e1f43d6734729b5cdbed242ebc3263ed
> refs/tags/1.1.2-rc2
> Configuration Matrix  gcc clang
> centos:7  --verbose --enable-libevent --enable-sslautotools   
>  
> 
>   
> 
> cmake 
>  
> 
>   
> 
> --verbose autotools   
>  
> 
>  
> 
> cmake 
>  
> 
>  
> 
> ubuntu:14.04  --verbose --enable-libevent --enable-sslautotools   
>  
> 
>   
>  
> 
> cmake 
>  
> 
>   
>  
> 
> --verbose autotools   
>  
> 
>  
>  
> 
> cmake 
>  
> 
>  
>  
> 
> 
> On Fri, May 12, 2017 at 8:07 AM, Alex Rukletsov  > wrote:
> Folks,
> 
> Please vote on releasing the following candidate as Apache Mesos 1.1.2.
> 
> 1.1.2 includes the following:
> 
> ** Bug
>   * [MESOS-2537] - AC_ARG_ENABLED checks are broken.
>   * [MESOS-5028] - Copy provisioner cannot replace directory with symlink.
>   * [MESOS-5172] - Registry puller cannot fetch blobs correctly from http
> Redirect 3xx urls.
>   * [MESOS-6327] - Large docker images causes container launch failures:
> Too many levels of symbolic links.
>   * [MESOS-7057] - Consider using the relink functionality of libprocess in
> the executor driver.
>   * [MESOS-7119] - Mesos master crash while accepting inverse offer.
>   * [MESOS-7152] - The agent may be flapping after the machine reboots due
> to provisioner recover.
>   * [MESOS-7197] - Requesting tiny amount of CPU crashes master.
>   * [MESOS-7210] - HTTP health check doesn't work when mesos runs with
> --docker_mesos_

New YouTube channel to house working group recordings?

[Judith Malnick]

Hi All,

I'd like to start an Apache Mesos YouTube channel to post the recordings of
public meetings and would also like to post the meetings on a new playlist
here , so
that community members in China can access the recordings without VPN.

Please let me know your thoughts.

Best,
Judith
-- 
Judith Malnick
DC/OS Community Manager
310-709-1517

[Containerization WG] Notes 05/18/2017

[Jie Yu]

Hi

We had our first WG meeting today. We went over the feature "wish list",
and tag them with different urgency and effort required. We tried to assign
high urgency features to folks in the working group and assign the
reviewers.

*We still need help on some of the items. So if you're willing to help,
please let us know (by replying this thread or reply to me).*

Please see the notes in the doc:
https://docs.google.com/document/d/1z55a7tLZFoRWVuUxz1FZwgxkHeugtc2nHR89skFXSpU/edit?usp=sharing

The recording can be found here:
https://drive.google.com/open?id=0B78XyMjTpvTmU0FSVExwSlBEdW8

- Jie

[GitHub] mesos pull request #213: Update vendored protobuf tar.gz to 3.2.0.

[zhitaoli]

GitHub user zhitaoli opened a pull request:

https://github.com/apache/mesos/pull/213

Update vendored protobuf tar.gz to 3.2.0.

The content of `3rdparty/protobuf-3.2.0.tar.gz` is generated by:
- On a Mac OS, download and untar protobuf v3.2.0 source at
  https://github.com/google/protobuf/archive/v3.2.0.tar.gz;
- Run `./autogen.sh`;
- Recompress and tar by `tar -czvf`.

Review: https://reviews.apache.org/r/58358

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhitaoli/mesos 
public/zhitao/protobuf_320_binary_only

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mesos/pull/213.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #213


commit 3e88e8afbcc8e93fe358fdaf3063acd06fc5c99f
Author: Zhitao Li 
Date:   2017-04-06T20:51:57Z

Update vendored protobuf tar.gz to 3.2.0.

The content of `3rdparty/protobuf-3.2.0.tar.gz` is generated by:
- On a Mac OS, download and untar protobuf v3.2.0 source at
  https://github.com/google/protobuf/archive/v3.2.0.tar.gz;
- Run `./autogen.sh`;
- Recompress and tar by `tar -czvf`.

Review: https://reviews.apache.org/r/58358




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] mesos pull request #208: Update vendored protobuf tar.gz to 3.2.0.

[zhitaoli]

Github user zhitaoli closed the pull request at:

https://github.com/apache/mesos/pull/208


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: mesos git commit: Updated the outdated network isolator configure flag.

[Neil Conway]

This commit enables the port mapping isolator by default. Was that
intended? Among other things, it breaks the build on OSX:

$ ../mesos/configure --disable-java --disable-python
[...]
configure: error: cannot build network isolator
---
Network isolator is only supported on Linux!
---

Neil

On Thu, May 18, 2017 at 8:34 AM,   wrote:
> Repository: mesos
> Updated Branches:
>   refs/heads/master 8c564db51 -> 20dee4190
>
>
> Updated the outdated network isolator configure flag.
>
> This patch updated the outdated network isolator configure flag to
> more descriptive port mapping isolator.
>
> Review: https://reviews.apache.org/r/59193/
>
>
> Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
> Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/20dee419
> Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/20dee419
> Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/20dee419
>
> Branch: refs/heads/master
> Commit: 20dee4190838b5bc7eb9cb524af413e3fe3fe082
> Parents: 8c564db
> Author: Tim Hansen 
> Authored: Wed May 17 14:47:59 2017 -0700
> Committer: Jie Yu 
> Committed: Thu May 18 08:33:54 2017 -0700
>
> --
>  configure.ac| 18 +++---
>  src/Makefile.am |  6 +++---
>  src/master/flags.cpp|  4 ++--
>  src/master/flags.hpp|  4 ++--
>  src/master/master.cpp   |  4 ++--
>  src/slave/constants.hpp |  2 +-
>  src/slave/containerizer/mesos/containerizer.cpp |  4 ++--
>  src/slave/flags.cpp |  4 ++--
>  src/slave/flags.hpp |  2 +-
>  src/tests/environment.cpp   |  8 
>  src/tests/master_tests.cpp  |  4 ++--
>  src/tests/mesos.cpp |  6 +++---
>  12 files changed, 35 insertions(+), 31 deletions(-)
> --
>
>
> http://git-wip-us.apache.org/repos/asf/mesos/blob/20dee419/configure.ac
> --
> diff --git a/configure.ac b/configure.ac
> index 8c17307..d523670 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -187,6 +187,11 @@ AC_ARG_ENABLE([debug],
>option won't change them]),
>[], [enable_debug=no])
>
> +AC_ARG_ENABLE([port-mapping-isolator],
> +  AS_HELP_STRING([--enable-port-mapping-isolator],
> + [enable port mapping isolator]),
> +  [], [enable_port_mapping_isolator=yes])
> +
>  AC_ARG_ENABLE([java],
>AS_HELP_STRING([--disable-java],
>   [don't build Java bindings]),
> @@ -334,12 +339,11 @@ AC_ARG_WITH([libprocess],
> [specify where to locate the libprocess library]),
>  [], [])
>
> -# TODO(MESOS-4991): Since network-isolator is an optional feature, it should
> -# be enabled with --enable-network-isolator.
>  AC_ARG_WITH([network-isolator],
>  AS_HELP_STRING([--with-network-isolator],
> [builds the network isolator]),
> -[], [with_network_isolator=no])
> +[AC_MSG_WARN(["--with-network-isolator is being depreciated, 
> please use --enable-port-mapping-isolator instead."])],
> +[enable_port_mapping_isolator=yes])
>
>  AC_ARG_WITH([nl],
>  AS_HELP_STRING([--with-nl=@<:@DIR@:>@],
> @@ -1270,7 +1274,7 @@ AM_CONDITIONAL([WITH_BUNDLED_LIBPROCESS], [test 
> "x$with_bundled_libprocess" = "x
>
>
>  # Perform necessary configuration for network isolator.
> -if test "x$with_network_isolator" = "xyes"; then
> +if test "x$enable_port_mapping_isolator" = "xyes"; then
>if test -n "`echo $with_nl`"; then
>  CPPFLAGS="-I${with_nl}/include/libnl3 $CPPFLAGS"
>  LDFLAGS="-L${with_nl}/lib $LDFLAGS"
> @@ -1342,11 +1346,11 @@ https://github.com/thom311/libnl/releases
>  ---
>])])
>
> -  AC_DEFINE([WITH_NETWORK_ISOLATOR])
> +  AC_DEFINE([ENABLE_PORT_MAPPING_ISOLATOR])
>  fi
>
> -AM_CONDITIONAL([WITH_NETWORK_ISOLATOR],
> -   [test "x$with_network_isolator" = "xyes"])
> +AM_CONDITIONAL([ENABLE_PORT_MAPPING_ISOLATOR],
> +   [test "x$enable_port_mapping_isolator" = "xyes"])
>
>
>  # If the user has asked not to include the bundled NVML headers for
>
> http://git-wip-us.apache.org/repos/asf/mesos/blob/20dee419/src/Makefile.am
> --
> diff --git a/src/Makefile.am b/src/Makefile.am
> index 434a00e..3e71393 100644
> --- a/src/Makefile.am
> +++ b/src/Makefil

Re: [VOTE] Release Apache Mesos 1.1.2 (rc2)

[Till Toenshoff]

+1

Ran on internal CI. All failures I saw were well documented flaky tests.

> On May 13, 2017, at 12:26 AM, Vinod Kone  wrote:
> 
> +1 (binding)
> 
> Ran on ASF CI. The one configuration that is in "red" is due to a known
> flaky issue with perf core dump during test suite teardown.
> 
> *Revision*: 37d98c55e1f43d6734729b5cdbed242ebc3263ed
> 
>   - refs/tags/1.1.2-rc2
> 
> Configuration Matrix gcc clang
> centos:7 --verbose --enable-libevent --enable-ssl autotools
> [image: Success]
> 
> [image: Not run]
> cmake
> [image: Success]
> 
> [image: Not run]
> --verbose autotools
> [image: Success]
> 
> [image: Not run]
> cmake
> [image: Success]
> 
> [image: Not run]
> ubuntu:14.04 --verbose --enable-libevent --enable-ssl autotools
> [image: Success]
> 
> [image: Failed]
> 
> cmake
> [image: Success]
> 
> [image: Success]
> 
> --verbose autotools
> [image: Success]
> 
> [image: Success]
> 
> cmake
> [image: Success]
> 
> [image: Success]
> 
> 
> On Fri, May 12, 2017 at 8:07 AM, Alex Rukletsov  wrote:
> 
>> Folks,
>> 
>> Please vote on releasing the following candidate as Apache Mesos 1.1.2.
>> 
>> 1.1.2 includes the following:
>> 
>> 
>> ** Bug
>>  * [MESOS-2537] - AC_ARG_ENABLED checks are broken.
>>  * [MESOS-5028] - Copy provisioner cannot replace directory with symlink.
>>  * [MESOS-5172] - Registry puller cannot fetch blobs correctly from http
>> Redirect 3xx urls.
>>  * [MESOS-6327] - Large docker images causes container launch failures:
>> Too many levels of symbolic links.
>>  * [MESOS-7057] - Consider using the relink functionality of libprocess in
>> the executor driver.
>>  * [MESOS-7119] - Mesos master crash while accepting inverse offer.
>>  * [MESOS-7152] - The agent may be flapping after the machine reboots due
>> to provisioner recover.
>>  * [MESOS-7197] - Requesting t