[jira] [Commented] (MESOS-2800) Rename Option::get(const T& _t) to getOrElse() and refactor the original function
[
https://issues.apache.org/jira/browse/MESOS-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578443#comment-14578443
]
Joris Van Remoortere commented on MESOS-2800:
-
My understanding is that we aim for only having 1 way to express something
unless absolutely necessary. That would suggest only having 1 function.
I think without looking at the comments at the function declaration site, it is
not directly obvious what this function should do. Therefore, unless someone
has a good reason not to, I think it is valuable to rename this function to
{{getOrElse}} since the burden is currently still low.
Feel free to add me as a reviewer for your patch :-)
> Rename Option::get(const T& _t) to getOrElse() and refactor the original
> function
>
>
> Key: MESOS-2800
> URL: https://issues.apache.org/jira/browse/MESOS-2800
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Reporter: Mark Wang
>Assignee: Mark Wang
>Priority: Minor
> Labels: newbie
>
> As suggested, if we want to change the name then we should refactor the
> original function as opposed to having 2 copies.
> If we did have 2 versions of the same function, would it make more sense to
> delegate one of them to the other.
> As of today, there is only one file need to be refactor:
> 3rdparty/libprocess/3rdparty/stout/include/stout/os/osx.hpp at line 151, 161
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1911) Create libevent-backed Socket implementation
[ https://issues.apache.org/jira/browse/MESOS-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van Remoortere updated MESOS-1911: Description: The poll implementation of the socket interface allows us to use regular sockets with the libevent library. However; this is not optimal. We can scale better if we use the libevent APIs in a libevent specific implementation. This is already being done for an SSL specific implementation in MESOS-1913. > Create libevent-backed Socket implementation > > > Key: MESOS-1911 > URL: https://issues.apache.org/jira/browse/MESOS-1911 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > Labels: libprocess > > The poll implementation of the socket interface allows us to use regular > sockets with the libevent library. However; this is not optimal. We can scale > better if we use the libevent APIs in a libevent specific implementation. > This is already being done for an SSL specific implementation in MESOS-1913. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1911) Create libevent-backed Socket implementation
[ https://issues.apache.org/jira/browse/MESOS-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578420#comment-14578420 ] Joris Van Remoortere commented on MESOS-1911: - This is for a non-ssl version of the socket interface that is specifically backed by libevent. It has been delayed as it is not critical for ssl; we can use the poll based implementation. Here is the original review: https://reviews.apache.org/r/28673 > Create libevent-backed Socket implementation > > > Key: MESOS-1911 > URL: https://issues.apache.org/jira/browse/MESOS-1911 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > Labels: libprocess > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2834) Support different perf output formats
Ian Downes created MESOS-2834: - Summary: Support different perf output formats Key: MESOS-2834 URL: https://issues.apache.org/jira/browse/MESOS-2834 Project: Mesos Issue Type: Bug Components: isolation Affects Versions: 0.22.1 Reporter: Ian Downes The output format of perf changes in 3.14 (inserting an additional field) and in again in 4.1 (appending additional) fields. See kernel commits: 410136f5dd96b6013fe6d1011b523b1c247e1ccb d73515c03c6a2706e088094ff6095a3abefd398b Update the perf::parse() function to understand all these formats. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2394) Create styleguide for documentation
[
https://issues.apache.org/jira/browse/MESOS-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2394:
---
Sprint: Mesosphere Sprint 12
> Create styleguide for documentation
> ---
>
> Key: MESOS-2394
> URL: https://issues.apache.org/jira/browse/MESOS-2394
> Project: Mesos
> Issue Type: Documentation
>Reporter: Joerg Schad
>Assignee: Joerg Schad
>Priority: Minor
>
> As of right now different pages in our documentation use quite different
> styles. Consider for example the different emphasis for NOTE:
> * {noformat}> NOTE:
> http://mesos.apache.org/documentation/latest/slave-recovery/{noformat}
> * {noformat}*NOTE*: http://mesos.apache.org/documentation/latest/upgrades/
> {noformat}
> Would be great to establish a common style for the documentation!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2073) Fetcher cache file verification, updating and invalidation
[ https://issues.apache.org/jira/browse/MESOS-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2073: --- Sprint: Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint 10, Mesosphere Sprint 11) > Fetcher cache file verification, updating and invalidation > -- > > Key: MESOS-2073 > URL: https://issues.apache.org/jira/browse/MESOS-2073 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave >Reporter: Bernd Mathiske >Assignee: Bernd Mathiske >Priority: Minor > Original Estimate: 96h > Remaining Estimate: 96h > > The other tickets in the fetcher cache epic do not necessitate a check sum > (e.g. MD5, SHA*) for files cached by the fetcher. Whereas such a check sum > could be used to verify whether the file arrived without unintended > alterations, it can first and foremost be employed to detect and trigger > updates. > Scenario: If a UIR is requested for fetching and the indicated download has > the same check sum as the cached file, then the cache file will be used and > the download forgone. If the check sum is different, then fetching proceeds > and the cached file gets replaced. > This capability will be indicated by an additional field in the URI protobuf. > Details TBD, i.e. to be discussed in comments below. > In addition to the above, even if the check sum is the same, we can support > voluntary cache file invalidation: a fresh download can be requested, or the > caching behavior can be revoked entirely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2500) Doxygen setup for libprocess
[ https://issues.apache.org/jira/browse/MESOS-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2500: --- Sprint: Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Sprint 10, Mesosphere Sprint 11) > Doxygen setup for libprocess > > > Key: MESOS-2500 > URL: https://issues.apache.org/jira/browse/MESOS-2500 > Project: Mesos > Issue Type: Documentation > Components: libprocess >Reporter: Bernd Mathiske >Assignee: Joerg Schad > Original Estimate: 48h > Remaining Estimate: 48h > > Goals: > - Initial doxygen setup. > - Enable interested developers to generate already available doxygen content > locally in their workspace and view it. > - Form the basis for future contributions of more doxygen content. > 1. Devise a way to use Doxygen with Mesos source code. (For example, solve > this by adding optional brew/apt-get installation to the "Getting Started" > doc.) > 2. Create a make target for libprocess documentation that can be manually > triggered. > 3. Create initial library top level documentation. > 4. Enhance one header file with Doxygen. Make sure the generated output has > all necessary links to navigate from the lib to the file and back, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2119) Add Socket tests
[ https://issues.apache.org/jira/browse/MESOS-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2119: --- Sprint: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Sprint 10, Mesosphere Sprint 11) > Add Socket tests > > > Key: MESOS-2119 > URL: https://issues.apache.org/jira/browse/MESOS-2119 > Project: Mesos > Issue Type: Task > Components: libprocess >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > > Add more Socket specific tests to get coverage while doing libev to libevent > (w and wo SSL) move -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1911) Create libevent-backed Socket implementation
[ https://issues.apache.org/jira/browse/MESOS-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578238#comment-14578238 ] Marco Massenzio commented on MESOS-1911: [~jvanremoortere] is this a dup of MESOS-1913? title is the same, no description, so real difficult to tell. If it's a dup - mind closing (or ping me and I'll do it) > Create libevent-backed Socket implementation > > > Key: MESOS-1911 > URL: https://issues.apache.org/jira/browse/MESOS-1911 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > Labels: libprocess > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2787) mesos-ps fails with "KeyError: 'mem_rss_bytes'"
[
https://issues.apache.org/jira/browse/MESOS-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Mahler updated MESOS-2787:
---
Shepherd: Benjamin Mahler
> mesos-ps fails with "KeyError: 'mem_rss_bytes'"
> ---
>
> Key: MESOS-2787
> URL: https://issues.apache.org/jira/browse/MESOS-2787
> Project: Mesos
> Issue Type: Bug
> Components: cli, python api, slave, statistics
>Affects Versions: 0.21.1, 0.22.0, 0.22.1
> Environment: ubuntu 14.04, mesos 0.21.1
>Reporter: weitao zhou
>Assignee: weitao zhou
> Labels: easyfix, patch
> Fix For: 0.23.0
>
>
> When I run the command *mesos ps --master=$MESOS-MASTER:5050*, the following
> exception raised:
> {code: title=exception.sh}
> File "/usr/bin/mesos-ps", line 232, in
> main()
> File "/usr/bin/mesos-ps", line 217, in main
> sys.stdout.write(columns[4].truncate(mem(task, statistics)))
> File "/usr/bin/mesos-ps", line 93, in mem
> mem_rss_bytes = entry['statistics']['mem_rss_bytes']
> KeyError: 'mem_rss_bytes'
> {code}
> I tracked it and found that some mesos slave nodes can't return the
> statistics yet now. In my special case, the disk volumn of some my mesos
> slaves is 100% full.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MESOS-2787) mesos-ps fails with "KeyError: 'mem_rss_bytes'"
[
https://issues.apache.org/jira/browse/MESOS-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Mahler resolved MESOS-2787.
Resolution: Fixed
Fix Version/s: 0.23.0
> mesos-ps fails with "KeyError: 'mem_rss_bytes'"
> ---
>
> Key: MESOS-2787
> URL: https://issues.apache.org/jira/browse/MESOS-2787
> Project: Mesos
> Issue Type: Bug
> Components: cli, python api, slave, statistics
>Affects Versions: 0.21.1, 0.22.0, 0.22.1
> Environment: ubuntu 14.04, mesos 0.21.1
>Reporter: weitao zhou
>Assignee: weitao zhou
> Labels: easyfix, patch
> Fix For: 0.23.0
>
>
> When I run the command *mesos ps --master=$MESOS-MASTER:5050*, the following
> exception raised:
> {code: title=exception.sh}
> File "/usr/bin/mesos-ps", line 232, in
> main()
> File "/usr/bin/mesos-ps", line 217, in main
> sys.stdout.write(columns[4].truncate(mem(task, statistics)))
> File "/usr/bin/mesos-ps", line 93, in mem
> mem_rss_bytes = entry['statistics']['mem_rss_bytes']
> KeyError: 'mem_rss_bytes'
> {code}
> I tracked it and found that some mesos slave nodes can't return the
> statistics yet now. In my special case, the disk volumn of some my mesos
> slaves is 100% full.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2787) mesos-ps fails with "KeyError: 'mem_rss_bytes'"
[
https://issues.apache.org/jira/browse/MESOS-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Mahler updated MESOS-2787:
---
Summary: mesos-ps fails with "KeyError: 'mem_rss_bytes'" (was: The
exception "KeyError: 'mem_rss_bytes'" is being raised when run command mesos ps
--master=<> )
{noformat}
commit 799a3c929a2b8a92b22a92022b0d229d677cac2c
Author: weitao zhou
Date: Mon Jun 8 18:51:37 2015 -0700
Fixed a KeyError exception in mesos-ps.
Review: https://reviews.apache.org/r/34886
{noformat}
> mesos-ps fails with "KeyError: 'mem_rss_bytes'"
> ---
>
> Key: MESOS-2787
> URL: https://issues.apache.org/jira/browse/MESOS-2787
> Project: Mesos
> Issue Type: Bug
> Components: cli, python api, slave, statistics
>Affects Versions: 0.21.1, 0.22.0, 0.22.1
> Environment: ubuntu 14.04, mesos 0.21.1
>Reporter: weitao zhou
>Assignee: weitao zhou
> Labels: easyfix, patch
> Fix For: 0.23.0
>
>
> When I run the command *mesos ps --master=$MESOS-MASTER:5050*, the following
> exception raised:
> {code: title=exception.sh}
> File "/usr/bin/mesos-ps", line 232, in
> main()
> File "/usr/bin/mesos-ps", line 217, in main
> sys.stdout.write(columns[4].truncate(mem(task, statistics)))
> File "/usr/bin/mesos-ps", line 93, in mem
> mem_rss_bytes = entry['statistics']['mem_rss_bytes']
> KeyError: 'mem_rss_bytes'
> {code}
> I tracked it and found that some mesos slave nodes can't return the
> statistics yet now. In my special case, the disk volumn of some my mesos
> slaves is 100% full.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2810) mesos-executor reimplements subprocess
[
https://issues.apache.org/jira/browse/MESOS-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578158#comment-14578158
]
Benjamin Mahler commented on MESOS-2810:
For posterity, myself, [~vinodkone], [~idownes] and [~jieyu] also happened to
be looking at this sometime before this ticket was filed. We should be able to
replace it, since this code came before Subprocess.
We were trying to figure out the reason for the {{setsid}}
[loop|https://github.com/apache/mesos/blob/9e7b890a917fcf0ac4cd1738f060ba97af847b65/src/launcher/executor.cpp#L209],
since it is a key difference in the implementation. This is really old code
from the svn days, the thread is
[here|http://marc.info/?l=mesos-dev&m=142539941428719&w=2] but doesn't reveal
any additional explanation. As far as we can tell from the POSIX [fork
spec|http://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html] the
{{setsid}} loop should not be necessary:
{quote}
The child process ID also shall not match any active process group ID.
{quote}
However,
[setsid|http://pubs.opengroup.org/onlinepubs/9699919799/functions/setsid.html]
seems to be a bit more ambiguous about it:
{quote}
EPERM
The calling process is already a process group leader, or the process group
ID of a process other than the calling process matches the process ID of the
calling process.
{quote}
> mesos-executor reimplements subprocess
> --
>
> Key: MESOS-2810
> URL: https://issues.apache.org/jira/browse/MESOS-2810
> Project: Mesos
> Issue Type: Improvement
> Components: slave
>Reporter: Cody Maloney
> Labels: mesosphere, newbie, tech-debt
>
> The launchTask method is a re-implementation of libprocess subprocess
> https://github.com/apache/mesos/blob/master/src/launcher/executor.cpp#L110
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2693) Printing a resource should show information about reservation, disk etc
[ https://issues.apache.org/jira/browse/MESOS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578148#comment-14578148 ] Benjamin Mahler commented on MESOS-2693: Is there any plan to make it parseable? It seems a bit unfortunate that we broke the ability to serialize and de-serialize and get the same result back. Also, I had to add an [extra flag|https://github.com/apache/mesos/blob/9e7b890a917fcf0ac4cd1738f060ba97af847b65/src/examples/no_executor_framework.cpp#L277] to the no_executor_framework, we'll probably want to add parsing before this trend proliferates further. :) > Printing a resource should show information about reservation, disk etc > --- > > Key: MESOS-2693 > URL: https://issues.apache.org/jira/browse/MESOS-2693 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: brian wickman > Labels: twitter > Fix For: 0.23.0 > > > While new fields like DiskInfo and ReservationInfo have been added to > Resource protobuf, the output stream operator hasn't been updated to show > these. This is valuable information to have in the logs during debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2776) Master should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578131#comment-14578131 ] Yan Xu commented on MESOS-2776: --- https://reviews.apache.org/r/35239/ > Master should expose metrics about oversubscribed resources > --- > > Key: MESOS-2776 > URL: https://issues.apache.org/jira/browse/MESOS-2776 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Yan Xu > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2537) AC_ARG_ENABLED checks are broken
[
https://issues.apache.org/jira/browse/MESOS-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573449#comment-14573449
]
James Peach edited comment on MESOS-2537 at 6/9/15 12:10 AM:
-
Initial change: https://reviews.apache.org/r/33752/
Updating bundled dependency detection: https://reviews.apache.org/r/35084/
Update for libprocess: https://reviews.apache.org/r/35234/
After these changes,
* \--enable-bundled and --disable-bundled sets the default response for all
bundled packages
* \-\-with-$PACKAGE=bundled or --without-package forces the use of a bundled
package
FWIW --without-package should disable the use of a package, not revert to a
bundled version, but this semantic is consistent with MESOS-1071.
There's a bunch of unusual behavior around gmock and gtest that doesn't make
sense to me. If anyone can explain what the right behaviors is in this case,
I'd be terribly grateful.
was (Author: jamespeach):
Initial change: https://reviews.apache.org/r/33752/
Updating bundled dependency detection: https://reviews.apache.org/r/35084/
Update for libprocess: https://reviews.apache.org/r/35234/
After these changes,
* \--enable-bundled and --disable-bundled sets the default response for all
bundled packages
* \--with-$PACKAGE=bundled or --without-package forces the use of a bundled
package
FWIW --without-package should disable the use of a package, not revert to a
bundled version, but this semantic is consistent with MESOS-1071.
There's a bunch of unusual behavior around gmock and gtest that doesn't make
sense to me. If anyone can explain what the right behaviors is in this case,
I'd be terribly grateful.
> AC_ARG_ENABLED checks are broken
>
>
> Key: MESOS-2537
> URL: https://issues.apache.org/jira/browse/MESOS-2537
> Project: Mesos
> Issue Type: Bug
> Components: build
>Reporter: James Peach
>Assignee: James Peach
>Priority: Minor
>
> In a number of places, the Mesos configure script passes "$foo=yes" to the
> 2nd argument of {{AC_ARG_ENABLED}}. However, the 2nd argument is invoked when
> the option is provided in any form, not just when the {{\--enable-foo}} form
> is used. One result of this is that {{\--disable-optimize}} doesn't work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2537) AC_ARG_ENABLED checks are broken
[
https://issues.apache.org/jira/browse/MESOS-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573449#comment-14573449
]
James Peach edited comment on MESOS-2537 at 6/9/15 12:10 AM:
-
Initial change: https://reviews.apache.org/r/33752/
Updating bundled dependency detection: https://reviews.apache.org/r/35084/
Update for libprocess: https://reviews.apache.org/r/35234/
After these changes,
* \--enable-bundled and --disable-bundled sets the default response for all
bundled packages
* \--with-$PACKAGE=bundled or --without-package forces the use of a bundled
package
FWIW --without-package should disable the use of a package, not revert to a
bundled version, but this semantic is consistent with MESOS-1071.
There's a bunch of unusual behavior around gmock and gtest that doesn't make
sense to me. If anyone can explain what the right behaviors is in this case,
I'd be terribly grateful.
was (Author: jamespeach):
Initial change: https://reviews.apache.org/r/33752/
Updating bundled dependency detection: https://reviews.apache.org/r/35084/
Update for libprocess: https://reviews.apache.org/r/35234/
After these changes,
* --enable-bundled and --disable-bundled sets the default response for all
bundled packages
* --with-$PACKAGE=bundled or --without-package forces the use of a bundled
package
FWIW --without-package should disable the use of a package, not revert to a
bundled version, but this semantic is consistent with MESOS-1071.
There's a bunch of unusual behavior around gmock and gtest that doesn't make
sense to me. If anyone can explain what the right behaviors is in this case,
I'd be terribly grateful.
> AC_ARG_ENABLED checks are broken
>
>
> Key: MESOS-2537
> URL: https://issues.apache.org/jira/browse/MESOS-2537
> Project: Mesos
> Issue Type: Bug
> Components: build
>Reporter: James Peach
>Assignee: James Peach
>Priority: Minor
>
> In a number of places, the Mesos configure script passes "$foo=yes" to the
> 2nd argument of {{AC_ARG_ENABLED}}. However, the 2nd argument is invoked when
> the option is provided in any form, not just when the {{\--enable-foo}} form
> is used. One result of this is that {{\--disable-optimize}} doesn't work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2537) AC_ARG_ENABLED checks are broken
[
https://issues.apache.org/jira/browse/MESOS-2537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573449#comment-14573449
]
James Peach edited comment on MESOS-2537 at 6/9/15 12:10 AM:
-
Initial change: https://reviews.apache.org/r/33752/
Updating bundled dependency detection: https://reviews.apache.org/r/35084/
Update for libprocess: https://reviews.apache.org/r/35234/
After these changes,
* --enable-bundled and --disable-bundled sets the default response for all
bundled packages
* --with-$PACKAGE=bundled or --without-package forces the use of a bundled
package
FWIW --without-package should disable the use of a package, not revert to a
bundled version, but this semantic is consistent with MESOS-1071.
There's a bunch of unusual behavior around gmock and gtest that doesn't make
sense to me. If anyone can explain what the right behaviors is in this case,
I'd be terribly grateful.
was (Author: jamespeach):
https://reviews.apache.org/r/33752/
https://reviews.apache.org/r/35084/
> AC_ARG_ENABLED checks are broken
>
>
> Key: MESOS-2537
> URL: https://issues.apache.org/jira/browse/MESOS-2537
> Project: Mesos
> Issue Type: Bug
> Components: build
>Reporter: James Peach
>Assignee: James Peach
>Priority: Minor
>
> In a number of places, the Mesos configure script passes "$foo=yes" to the
> 2nd argument of {{AC_ARG_ENABLED}}. However, the 2nd argument is invoked when
> the option is provided in any form, not just when the {{\--enable-foo}} form
> is used. One result of this is that {{\--disable-optimize}} doesn't work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1104) Move linux/fs.hpp out of `mesos` namespace in linux/fs.h
[
https://issues.apache.org/jira/browse/MESOS-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-1104:
---
Summary: Move linux/fs.hpp out of `mesos` namespace in linux/fs.h (was:
Move linux/fs.hpp out of {{mesos}} namespace in linux/fs.h)
> Move linux/fs.hpp out of `mesos` namespace in linux/fs.h
>
>
> Key: MESOS-1104
> URL: https://issues.apache.org/jira/browse/MESOS-1104
> Project: Mesos
> Issue Type: Improvement
>Reporter: Archana kumari
>Assignee: Jojy Varghese
> Labels: newbie
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1104) Move linux/fs.hpp out of {{mesos}} namespace in linux/fs.h
[
https://issues.apache.org/jira/browse/MESOS-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-1104:
---
Summary: Move linux/fs.hpp out of {{mesos}} namespace in linux/fs.h (was:
TODO(benh): Move linux/fs.hpp out of 'mesos- namespace in linux/fs.h)
> Move linux/fs.hpp out of {{mesos}} namespace in linux/fs.h
> --
>
> Key: MESOS-1104
> URL: https://issues.apache.org/jira/browse/MESOS-1104
> Project: Mesos
> Issue Type: Improvement
>Reporter: Archana kumari
>Assignee: Jojy Varghese
> Labels: newbie
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-2833) Consolidate methods in MesosTest class.
Bartek Plotka created MESOS-2833: Summary: Consolidate methods in MesosTest class. Key: MESOS-2833 URL: https://issues.apache.org/jira/browse/MESOS-2833 Project: Mesos Issue Type: Improvement Components: test Reporter: Bartek Plotka Priority: Minor Some of overloaded methods in MesosTest (tests/mesos.hpp) are growing a bit out of hand. (e.g. StartSlave) Let's consolidate these somehow - for example using Option<>. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2035) Add reason to containerizer proto Termination
[ https://issues.apache.org/jira/browse/MESOS-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578027#comment-14578027 ] Jie Yu commented on MESOS-2035: --- cc [~tnachen] [~idownes] > Add reason to containerizer proto Termination > - > > Key: MESOS-2035 > URL: https://issues.apache.org/jira/browse/MESOS-2035 > Project: Mesos > Issue Type: Improvement > Components: slave >Affects Versions: 0.21.0 >Reporter: Dominic Hamon >Assignee: Joerg Schad >Priority: Critical > > When an isolator kills a task, the reason is unknown. As part of MESOS-1830, > the reason is set to a general one but ideally we would have the termination > reason to pass through to the status update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2035) Add reason to containerizer proto Termination
[ https://issues.apache.org/jira/browse/MESOS-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Yu updated MESOS-2035: -- Priority: Critical (was: Minor) > Add reason to containerizer proto Termination > - > > Key: MESOS-2035 > URL: https://issues.apache.org/jira/browse/MESOS-2035 > Project: Mesos > Issue Type: Improvement > Components: slave >Affects Versions: 0.21.0 >Reporter: Dominic Hamon >Assignee: Joerg Schad >Priority: Critical > > When an isolator kills a task, the reason is unknown. As part of MESOS-1830, > the reason is set to a general one but ideally we would have the termination > reason to pass through to the status update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2035) Add reason to containerizer proto Termination
[ https://issues.apache.org/jira/browse/MESOS-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578025#comment-14578025 ] Jie Yu commented on MESOS-2035: --- This problem pops again when we are implementing oversubscription. See MESOS-2653 and https://reviews.apache.org/r/34720/ for details. Here is my proposal for solving this issue: 1) We add a TaskStatus::Reason field in containerizer::Termination protobuf (and deprecate the 'killed' field) 2) In slave's per executor data structure (struct Executor), we maintain an optional 'reason' field. When the slave destroys a container (e.g., due to registration timeout, failed to set resource limits, failed to launch container, qos controller kill, etc.), it will save the 'reason' field in struct Executor. 3) Containerizer is responsible for setting the 'reason' field inside containerizer::Termination (e.g., REASON_MEMORY_LIMIT, REASON_DISK_LIMIT, etc.) 4) In sendExecutorTerminatedStatusUpdate, we look at both reasons (one from slave's executor data structure and one from Termination protobuf). The current proposal is to prefer the reason from Termination protobuf. But in the future, when we allow multiple reasons to be sent (MESOS-2657), we can send both to the scheduler. > Add reason to containerizer proto Termination > - > > Key: MESOS-2035 > URL: https://issues.apache.org/jira/browse/MESOS-2035 > Project: Mesos > Issue Type: Improvement > Components: slave >Affects Versions: 0.21.0 >Reporter: Dominic Hamon >Assignee: Joerg Schad >Priority: Minor > > When an isolator kills a task, the reason is unknown. As part of MESOS-1830, > the reason is set to a general one but ideally we would have the termination > reason to pass through to the status update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2035) Add reason to containerizer proto Termination
[ https://issues.apache.org/jira/browse/MESOS-2035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Yu updated MESOS-2035: -- Target Version/s: 0.23.0 > Add reason to containerizer proto Termination > - > > Key: MESOS-2035 > URL: https://issues.apache.org/jira/browse/MESOS-2035 > Project: Mesos > Issue Type: Improvement > Components: slave >Affects Versions: 0.21.0 >Reporter: Dominic Hamon >Assignee: Joerg Schad >Priority: Minor > > When an isolator kills a task, the reason is unknown. As part of MESOS-1830, > the reason is set to a general one but ideally we would have the termination > reason to pass through to the status update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-999) Slave should wait() and start executor registration timeout after launch
[ https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14578018#comment-14578018 ] Ian Downes commented on MESOS-999: -- cc [~xujyan] > Slave should wait() and start executor registration timeout after launch > - > > Key: MESOS-999 > URL: https://issues.apache.org/jira/browse/MESOS-999 > Project: Mesos > Issue Type: Bug > Components: isolation >Affects Versions: 0.18.0 >Reporter: Ian Downes >Assignee: Nishant Suneja >Priority: Minor > > The current code will start launch a container and wait on it before the > launch is complete. We should do this only after the container has > successfully launched. Likewise for the executor registration timeout. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2555) Document problem/solution of MESOS-2419 in documentation.
[ https://issues.apache.org/jira/browse/MESOS-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2555: --- Story Points: 1 > Document problem/solution of MESOS-2419 in documentation. > - > > Key: MESOS-2555 > URL: https://issues.apache.org/jira/browse/MESOS-2555 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad >Priority: Critical > > As the problem encountered in MESOS-2419 is a common problem with the default > systemd configuration it would make sense to document this in the upgrade > guide or somewhere else in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2555) Document problem/solution of MESOS-2419 in documentation.
[ https://issues.apache.org/jira/browse/MESOS-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2555: --- Sprint: Mesosphere Sprint 12 > Document problem/solution of MESOS-2419 in documentation. > - > > Key: MESOS-2555 > URL: https://issues.apache.org/jira/browse/MESOS-2555 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad >Priority: Critical > > As the problem encountered in MESOS-2419 is a common problem with the default > systemd configuration it would make sense to document this in the upgrade > guide or somewhere else in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2555) Document problem/solution of MESOS-2419 in documentation.
[ https://issues.apache.org/jira/browse/MESOS-2555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577983#comment-14577983 ] Marco Massenzio commented on MESOS-2555: can we please change the title of this story in a way that makes sense to people (without having to parse MESOS-2419 too?) Also, I notice the review was posted back in March - any progress? > Document problem/solution of MESOS-2419 in documentation. > - > > Key: MESOS-2555 > URL: https://issues.apache.org/jira/browse/MESOS-2555 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad >Assignee: Joerg Schad >Priority: Critical > > As the problem encountered in MESOS-2419 is a common problem with the default > systemd configuration it would make sense to document this in the upgrade > guide or somewhere else in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2157) Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
[
https://issues.apache.org/jira/browse/MESOS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577979#comment-14577979
]
Marco Massenzio commented on MESOS-2157:
I just noticed a few things:
1. there is a patch that was committed? (ref'd halfway through the comments);
2. this should have perhaps been split into two (or possibly, three - given
[~adam-mesos] comment) separate stories?
> Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
>
>
> Key: MESOS-2157
> URL: https://issues.apache.org/jira/browse/MESOS-2157
> Project: Mesos
> Issue Type: Task
> Components: master
>Reporter: Niklas Quarfot Nielsen
>Assignee: Alexander Rojas
>Priority: Trivial
> Labels: mesosphere, newbie
>
> master/state.json exports the entire state of the cluster and can, for large
> clusters, become massive (tens of megabytes of JSON).
> Often, a client only need information about subsets of the entire state, for
> example all connected slaves, or information (registration info, tasks, etc)
> belonging to a particular framework.
> We can partition state.json into many smaller endpoints, but for starters,
> being able to get slave information and tasks information per framework would
> be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2559) Do not use RunTaskMessage.framework_id.
[
https://issues.apache.org/jira/browse/MESOS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577974#comment-14577974
]
Marco Massenzio commented on MESOS-2559:
Keeping in the backlog as, per the comment in the code review, it would seem
that it will have to wait for {{0.23}} to ship before it can be applied.
Also, moving it back to {{Open}} state, as there is a {{TODO}} in the Tests
section that, clearly requires work to do.
> Do not use RunTaskMessage.framework_id.
> ---
>
> Key: MESOS-2559
> URL: https://issues.apache.org/jira/browse/MESOS-2559
> Project: Mesos
> Issue Type: Bug
> Components: framework
>Reporter: Kapil Arya
>Assignee: Kapil Arya
>
> Assume that FrameworkInfo.id is always set and so need to read/set
> RunTaskMessage.framework_id.
> This should land after https://issues.apache.org/jira/browse/MESOS-2558 has
> been shipped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2559) Do not use RunTaskMessage.framework_id.
[ https://issues.apache.org/jira/browse/MESOS-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2559: --- Story Points: 1 > Do not use RunTaskMessage.framework_id. > --- > > Key: MESOS-2559 > URL: https://issues.apache.org/jira/browse/MESOS-2559 > Project: Mesos > Issue Type: Bug > Components: framework >Reporter: Kapil Arya >Assignee: Kapil Arya > > Assume that FrameworkInfo.id is always set and so need to read/set > RunTaskMessage.framework_id. > This should land after https://issues.apache.org/jira/browse/MESOS-2558 has > been shipped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2708: --- Story Points: 2 > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Alexander Rojas > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2708) Design doc for the Executor HTTP API
[ https://issues.apache.org/jira/browse/MESOS-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2708: --- Sprint: Mesosphere Sprint 12 > Design doc for the Executor HTTP API > > > Key: MESOS-2708 > URL: https://issues.apache.org/jira/browse/MESOS-2708 > Project: Mesos > Issue Type: Bug >Reporter: Alexander Rojas >Assignee: Alexander Rojas > > This tracks the design of the Executor HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2756) Update style guide: Avoid object slicing
[
https://issues.apache.org/jira/browse/MESOS-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2756:
---
Story Points: 1
> Update style guide: Avoid object slicing
>
>
> Key: MESOS-2756
> URL: https://issues.apache.org/jira/browse/MESOS-2756
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joris Van Remoortere
>Assignee: Joris Van Remoortere
> Labels: c++
>
> In order to improve the safety of our code base, let's augment the style
> guide to:
> "Disallow public construction of base classes"
> so that we can avoid the object slicing problem. This is a good pattern to
> follow in general as it prevents subtle semantic bugs like the following:
> {code:title=ObjectSlicing.cpp|borderStyle=solid}
> #include
> #include
> class Base {
> public:
> Base(int _v) : v(_v) {}
> virtual int get() const { return v; }
> protected:
> int v;
> };
> class Derived : public Base {
> public:
> Derived(int _v) : Base(_v) {}
> virtual int get() const { return v + 1; }
> };
> int main() {
> Base b(5);
> Derived d(5);
> std::vector vec;
> vec.push_back(b);
> vec.push_back(d);
> for (const auto& v : vec) {
> printf("[%d]\n", v.get());
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2756) Update style guide: Avoid object slicing
[
https://issues.apache.org/jira/browse/MESOS-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2756:
---
Sprint: Mesosphere Sprint 12
> Update style guide: Avoid object slicing
>
>
> Key: MESOS-2756
> URL: https://issues.apache.org/jira/browse/MESOS-2756
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joris Van Remoortere
>Assignee: Joris Van Remoortere
> Labels: c++
>
> In order to improve the safety of our code base, let's augment the style
> guide to:
> "Disallow public construction of base classes"
> so that we can avoid the object slicing problem. This is a good pattern to
> follow in general as it prevents subtle semantic bugs like the following:
> {code:title=ObjectSlicing.cpp|borderStyle=solid}
> #include
> #include
> class Base {
> public:
> Base(int _v) : v(_v) {}
> virtual int get() const { return v; }
> protected:
> int v;
> };
> class Derived : public Base {
> public:
> Derived(int _v) : Base(_v) {}
> virtual int get() const { return v + 1; }
> };
> int main() {
> Base b(5);
> Derived d(5);
> std::vector vec;
> vec.push_back(b);
> vec.push_back(d);
> for (const auto& v : vec) {
> printf("[%d]\n", v.get());
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2464) Authentication failure may lead to slave crash
[
https://issues.apache.org/jira/browse/MESOS-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2464:
---
Sprint: Mesosphere Sprint 12
> Authentication failure may lead to slave crash
> --
>
> Key: MESOS-2464
> URL: https://issues.apache.org/jira/browse/MESOS-2464
> Project: Mesos
> Issue Type: Bug
>Reporter: Till Toenshoff
>Assignee: Till Toenshoff
> Fix For: 0.23.0
>
>
> When slave authentication fails, the following attempt to transmit a
> {{UnregisterSlaveMessage}} may cause a crash within the slave.
> {noformat}
> E0309 01:08:34.819758 336699392 slave.cpp:740] Master
> master@192.168.178.20:5050 refused authentication
> I0309 01:08:34.819787 336699392 slave.cpp:538] Master refused authentication;
> unregistering and shutting down
> [libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> libprocess: slave(1)@192.168.178.20:5051 terminating due to CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> {noformat}
> The problem here is the following code:
> {noformat}
> UnregisterSlaveMessage message_;
> message_.mutable_slave_id()->MergeFrom(info.id());
> {noformat}
> Authentication happens before registration. {{info.id}} is an optional member
> (of {{SlaveInfo}}) and not known yet. It is set later, while registering. So
> {{slave_id}} will remain unset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2464) Authentication failure may lead to slave crash
[
https://issues.apache.org/jira/browse/MESOS-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2464:
---
Story Points: 1
> Authentication failure may lead to slave crash
> --
>
> Key: MESOS-2464
> URL: https://issues.apache.org/jira/browse/MESOS-2464
> Project: Mesos
> Issue Type: Bug
>Reporter: Till Toenshoff
>Assignee: Till Toenshoff
> Fix For: 0.23.0
>
>
> When slave authentication fails, the following attempt to transmit a
> {{UnregisterSlaveMessage}} may cause a crash within the slave.
> {noformat}
> E0309 01:08:34.819758 336699392 slave.cpp:740] Master
> master@192.168.178.20:5050 refused authentication
> I0309 01:08:34.819787 336699392 slave.cpp:538] Master refused authentication;
> unregistering and shutting down
> [libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> libprocess: slave(1)@192.168.178.20:5051 terminating due to CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> {noformat}
> The problem here is the following code:
> {noformat}
> UnregisterSlaveMessage message_;
> message_.mutable_slave_id()->MergeFrom(info.id());
> {noformat}
> Authentication happens before registration. {{info.id}} is an optional member
> (of {{SlaveInfo}}) and not known yet. It is set later, while registering. So
> {{slave_id}} will remain unset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2464) Authentication failure may lead to slave crash
[
https://issues.apache.org/jira/browse/MESOS-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577953#comment-14577953
]
Marco Massenzio commented on MESOS-2464:
This review was last commented by [~vi...@twitter.com] more than a couple of
months back:
{noformat}
Posted 2 months, 1 week ago (March 30, 2015, 5:58 p.m.)
{noformat}
but there seems to have been no activity since?
The diff (and the suggested fix) seems pretty simple to achieve: can we please
progress this to some resolution, so we can close?
Thanks!
> Authentication failure may lead to slave crash
> --
>
> Key: MESOS-2464
> URL: https://issues.apache.org/jira/browse/MESOS-2464
> Project: Mesos
> Issue Type: Bug
>Reporter: Till Toenshoff
>Assignee: Till Toenshoff
> Fix For: 0.23.0
>
>
> When slave authentication fails, the following attempt to transmit a
> {{UnregisterSlaveMessage}} may cause a crash within the slave.
> {noformat}
> E0309 01:08:34.819758 336699392 slave.cpp:740] Master
> master@192.168.178.20:5050 refused authentication
> I0309 01:08:34.819787 336699392 slave.cpp:538] Master refused authentication;
> unregistering and shutting down
> [libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> libprocess: slave(1)@192.168.178.20:5051 terminating due to CHECK failed:
> IsInitialized(): Can't serialize message of type
> "mesos.internal.UnregisterSlaveMessage" because it is missing required
> fields: slave_id.value
> {noformat}
> The problem here is the following code:
> {noformat}
> UnregisterSlaveMessage message_;
> message_.mutable_slave_id()->MergeFrom(info.id());
> {noformat}
> Authentication happens before registration. {{info.id}} is an optional member
> (of {{SlaveInfo}}) and not known yet. It is set later, while registering. So
> {{slave_id}} will remain unset.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1257) ExternalContainerizerTest.Launch is flaky
[
https://issues.apache.org/jira/browse/MESOS-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577946#comment-14577946
]
Marco Massenzio commented on MESOS-1257:
Looks like this is not being worked on?
The [review|https://reviews.apache.org/r/21695/] referenced above has not been
updated in a year (and it's also, according to our bot, a "bad patch"):
{noformat}
Posted 1 year ago (May 28, 2014, 2:52 p.m.)
{noformat}
[~tillt] please correct if I'm wrong, I'm moving this back to the {{Open}}
state (but keeping in backlog).
> ExternalContainerizerTest.Launch is flaky
> -
>
> Key: MESOS-1257
> URL: https://issues.apache.org/jira/browse/MESOS-1257
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Till Toenshoff
> Labels: flaky
>
> {noformat}
> [--] 1 test from ExternalContainerizerTest
> [ RUN ] ExternalContainerizerTest.Launch
> 2014-04-28
> 17:04:14,345:44634(0x7f94deffd700):ZOO_ERROR@handle_socket_error_msg@1697:
> Socket [127.0.0.1:52535] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2014-04-28
> 17:04:17,681:44634(0x7f94deffd700):ZOO_ERROR@handle_socket_error_msg@1697:
> Socket [127.0.0.1:52535] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> 2014-04-28
> 17:04:21,018:44634(0x7f94deffd700):ZOO_ERROR@handle_socket_error_msg@1697:
> Socket [127.0.0.1:52535] zk retcode=-4, errno=111(Connection refused): server
> refused to accept the client
> ../../../mesos/src/tests/external_containerizer_test.cpp:202: Failure
> Failed to wait 10secs for status
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2752) Add HTB queueing discipline wrapper class
[ https://issues.apache.org/jira/browse/MESOS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577931#comment-14577931 ] Paul Brett commented on MESOS-2752: --- https://reviews.apache.org/r/35225/ > Add HTB queueing discipline wrapper class > - > > Key: MESOS-2752 > URL: https://issues.apache.org/jira/browse/MESOS-2752 > Project: Mesos > Issue Type: Improvement >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > Network isolator uses a Hierarchical Token Bucket (HTB) traffic control > discipline on the egress filter inside each container as the root for adding > traffic filters. A HTB wrapper is needed to access the network statistics > for this interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1977) Disk Isolator Usage Metrics
[
https://issues.apache.org/jira/browse/MESOS-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577903#comment-14577903
]
Marco Massenzio commented on MESOS-1977:
This patch is marked as "discarded" - moving this story to the {{Open}} state;
please update accordingly if you are still working on it.
> Disk Isolator Usage Metrics
>
>
> Key: MESOS-1977
> URL: https://issues.apache.org/jira/browse/MESOS-1977
> Project: Mesos
> Issue Type: Improvement
> Components: isolation
>Reporter: Joris Van Remoortere
>Assignee: Joris Van Remoortere
>
> Implement just the usage statistics aspect of the block io isolator for the
> mesos containerizer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1911) Create libevent-backed Socket implementation
[ https://issues.apache.org/jira/browse/MESOS-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577900#comment-14577900 ] Marco Massenzio commented on MESOS-1911: Also a description and effort estimate (points), please. > Create libevent-backed Socket implementation > > > Key: MESOS-1911 > URL: https://issues.apache.org/jira/browse/MESOS-1911 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > Labels: libprocess > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2157) Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
[
https://issues.apache.org/jira/browse/MESOS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2157:
---
Story Points: 5
> Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
>
>
> Key: MESOS-2157
> URL: https://issues.apache.org/jira/browse/MESOS-2157
> Project: Mesos
> Issue Type: Task
> Components: master
>Reporter: Niklas Quarfot Nielsen
>Assignee: Alexander Rojas
>Priority: Trivial
> Labels: mesosphere, newbie
>
> master/state.json exports the entire state of the cluster and can, for large
> clusters, become massive (tens of megabytes of JSON).
> Often, a client only need information about subsets of the entire state, for
> example all connected slaves, or information (registration info, tasks, etc)
> belonging to a particular framework.
> We can partition state.json into many smaller endpoints, but for starters,
> being able to get slave information and tasks information per framework would
> be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-708) Static files missing "Last-Modified" HTTP headers
[ https://issues.apache.org/jira/browse/MESOS-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-708: -- Story Points: 2 > Static files missing "Last-Modified" HTTP headers > - > > Key: MESOS-708 > URL: https://issues.apache.org/jira/browse/MESOS-708 > Project: Mesos > Issue Type: Improvement > Components: libprocess, webui >Affects Versions: 0.13.0 >Reporter: Ross Allen >Assignee: Alexander Rojas > Labels: mesosphere > > Static assets served by the Mesos master don't return "Last-Modified" HTTP > headers. That means clients receive a 200 status code and re-download assets > on every page request even if the assets haven't changed. Because Angular JS > does most of the work, the downloading happens only when you navigate to > Mesos master in your browser or use the browser's refresh. > Example header for "mesos.css": > HTTP/1.1 200 OK > Date: Thu, 26 Sep 2013 17:18:52 GMT > Content-Length: 1670 > Content-Type: text/css > Clients sometimes use the "Date" header for the same effect as > "Last-Modified", but the date is always the time of the response from the > server, i.e. it changes on every request and makes the assets look new every > time. > The "Last-Modified" header should be added and should be the last modified > time of the file. On subsequent requests for the same files, the master > should return 304 responses with no content rather than 200 with the full > files. It could save clients a lot of download time since Mesos assets are > rather heavyweight. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-708) Static files missing "Last-Modified" HTTP headers
[ https://issues.apache.org/jira/browse/MESOS-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577898#comment-14577898 ] Marco Massenzio commented on MESOS-708: --- This was last updated in Feb - what's going on here? Keeping in backlog for now. > Static files missing "Last-Modified" HTTP headers > - > > Key: MESOS-708 > URL: https://issues.apache.org/jira/browse/MESOS-708 > Project: Mesos > Issue Type: Improvement > Components: libprocess, webui >Affects Versions: 0.13.0 >Reporter: Ross Allen >Assignee: Alexander Rojas > Labels: mesosphere > > Static assets served by the Mesos master don't return "Last-Modified" HTTP > headers. That means clients receive a 200 status code and re-download assets > on every page request even if the assets haven't changed. Because Angular JS > does most of the work, the downloading happens only when you navigate to > Mesos master in your browser or use the browser's refresh. > Example header for "mesos.css": > HTTP/1.1 200 OK > Date: Thu, 26 Sep 2013 17:18:52 GMT > Content-Length: 1670 > Content-Type: text/css > Clients sometimes use the "Date" header for the same effect as > "Last-Modified", but the date is always the time of the response from the > server, i.e. it changes on every request and makes the assets look new every > time. > The "Last-Modified" header should be added and should be the last modified > time of the file. On subsequent requests for the same files, the master > should return 304 responses with no content rather than 200 with the full > files. It could save clients a lot of download time since Mesos assets are > rather heavyweight. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2157) Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
[
https://issues.apache.org/jira/browse/MESOS-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577896#comment-14577896
]
Marco Massenzio commented on MESOS-2157:
[~arojas] Can you please update this story?
Is this still going to be committed, or should it moved back from
{{Reviewable}}?
Also, I'm estimating it at 5 points (the time the review just sat there, should
not count towards this) but please let me know if it was much more effort to
implement.
Keeping in the Backlog for now.
> Add /master/slaves and /master/frameworks/{framework}/tasks/{task} endpoints
>
>
> Key: MESOS-2157
> URL: https://issues.apache.org/jira/browse/MESOS-2157
> Project: Mesos
> Issue Type: Task
> Components: master
>Reporter: Niklas Quarfot Nielsen
>Assignee: Alexander Rojas
>Priority: Trivial
> Labels: mesosphere, newbie
>
> master/state.json exports the entire state of the cluster and can, for large
> clusters, become massive (tens of megabytes of JSON).
> Often, a client only need information about subsets of the entire state, for
> example all connected slaves, or information (registration info, tasks, etc)
> belonging to a particular framework.
> We can partition state.json into many smaller endpoints, but for starters,
> being able to get slave information and tasks information per framework would
> be useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1303) ExamplesTest.{TestFramework, NoExecutorFramework} flaky
[
https://issues.apache.org/jira/browse/MESOS-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-1303:
---
Sprint: Mesosphere Sprint 12
> ExamplesTest.{TestFramework, NoExecutorFramework} flaky
> ---
>
> Key: MESOS-1303
> URL: https://issues.apache.org/jira/browse/MESOS-1303
> Project: Mesos
> Issue Type: Bug
> Components: test
>Reporter: Ian Downes
>Assignee: Till Toenshoff
> Labels: flaky
>
> I'm having trouble reproducing this but I did observe it once on my OSX
> system:
> {noformat}
> [==] Running 2 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 2 tests from ExamplesTest
> [ RUN ] ExamplesTest.TestFramework
> ../../src/tests/script.cpp:81: Failure
> Failed
> test_framework_test.sh terminated with signal 'Abort trap: 6'
> [ FAILED ] ExamplesTest.TestFramework (953 ms)
> [ RUN ] ExamplesTest.NoExecutorFramework
> [ OK ] ExamplesTest.NoExecutorFramework (10162 ms)
> [--] 2 tests from ExamplesTest (5 ms total)
> [--] Global test environment tear-down
> [==] 2 tests from 1 test case ran. (11121 ms total)
> [ PASSED ] 1 test.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] ExamplesTest.TestFramework
> {noformat}
> when investigating a failed make check for https://reviews.apache.org/r/20971/
> {noformat}
> [--] 6 tests from ExamplesTest
> [ RUN ] ExamplesTest.TestFramework
> [ OK ] ExamplesTest.TestFramework (8643 ms)
> [ RUN ] ExamplesTest.NoExecutorFramework
> tests/script.cpp:81: Failure
> Failed
> no_executor_framework_test.sh terminated with signal 'Aborted'
> [ FAILED ] ExamplesTest.NoExecutorFramework (7220 ms)
> [ RUN ] ExamplesTest.JavaFramework
> [ OK ] ExamplesTest.JavaFramework (11181 ms)
> [ RUN ] ExamplesTest.JavaException
> [ OK ] ExamplesTest.JavaException (5624 ms)
> [ RUN ] ExamplesTest.JavaLog
> [ OK ] ExamplesTest.JavaLog (6472 ms)
> [ RUN ] ExamplesTest.PythonFramework
> [ OK ] ExamplesTest.PythonFramework (14467 ms)
> [--] 6 tests from ExamplesTest (53607 ms total)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1303) ExamplesTest.{TestFramework, NoExecutorFramework} flaky
[
https://issues.apache.org/jira/browse/MESOS-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-1303:
---
Story Points: 1
> ExamplesTest.{TestFramework, NoExecutorFramework} flaky
> ---
>
> Key: MESOS-1303
> URL: https://issues.apache.org/jira/browse/MESOS-1303
> Project: Mesos
> Issue Type: Bug
> Components: test
>Reporter: Ian Downes
>Assignee: Till Toenshoff
> Labels: flaky
>
> I'm having trouble reproducing this but I did observe it once on my OSX
> system:
> {noformat}
> [==] Running 2 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 2 tests from ExamplesTest
> [ RUN ] ExamplesTest.TestFramework
> ../../src/tests/script.cpp:81: Failure
> Failed
> test_framework_test.sh terminated with signal 'Abort trap: 6'
> [ FAILED ] ExamplesTest.TestFramework (953 ms)
> [ RUN ] ExamplesTest.NoExecutorFramework
> [ OK ] ExamplesTest.NoExecutorFramework (10162 ms)
> [--] 2 tests from ExamplesTest (5 ms total)
> [--] Global test environment tear-down
> [==] 2 tests from 1 test case ran. (11121 ms total)
> [ PASSED ] 1 test.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] ExamplesTest.TestFramework
> {noformat}
> when investigating a failed make check for https://reviews.apache.org/r/20971/
> {noformat}
> [--] 6 tests from ExamplesTest
> [ RUN ] ExamplesTest.TestFramework
> [ OK ] ExamplesTest.TestFramework (8643 ms)
> [ RUN ] ExamplesTest.NoExecutorFramework
> tests/script.cpp:81: Failure
> Failed
> no_executor_framework_test.sh terminated with signal 'Aborted'
> [ FAILED ] ExamplesTest.NoExecutorFramework (7220 ms)
> [ RUN ] ExamplesTest.JavaFramework
> [ OK ] ExamplesTest.JavaFramework (11181 ms)
> [ RUN ] ExamplesTest.JavaException
> [ OK ] ExamplesTest.JavaException (5624 ms)
> [ RUN ] ExamplesTest.JavaLog
> [ OK ] ExamplesTest.JavaLog (6472 ms)
> [ RUN ] ExamplesTest.PythonFramework
> [ OK ] ExamplesTest.PythonFramework (14467 ms)
> [--] 6 tests from ExamplesTest (53607 ms total)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2743) Include ExecutorInfos in master/state.json
[ https://issues.apache.org/jira/browse/MESOS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2743: --- Sprint: Mesosphere Sprint 12 > Include ExecutorInfos in master/state.json > -- > > Key: MESOS-2743 > URL: https://issues.apache.org/jira/browse/MESOS-2743 > Project: Mesos > Issue Type: Improvement > Components: json api >Reporter: Adam B >Assignee: haosdent > Labels: mesosphere > > The slave/state.json already reports executorInfos: > https://github.com/apache/mesos/blob/0.22.1/src/slave/http.cpp#L215-219 > Would be great to see this in the master/state.json as well, so external > tools don't have to query each slave to find out executor resources, sandbox > directories, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2832) Enable configuring Mesos with environment variables without having them leak to tasks launched
Cody Maloney created MESOS-2832: --- Summary: Enable configuring Mesos with environment variables without having them leak to tasks launched Key: MESOS-2832 URL: https://issues.apache.org/jira/browse/MESOS-2832 Project: Mesos Issue Type: Wish Reporter: Cody Maloney Priority: Critical Currently if mesos is configured with environment variables (MESOS_MODULES), those show up in every task which is launched unless the executor explicitly cleans them up. If the task being launched happens to be something libprocess / mesos based, this can often prevent the task from starting up (A scheduler has issues loading a module intended for the slave). There are also cases where it would be nice to be able to change what the PATH is that tasks launch with (the host may have more in the path than tasks are supposed to / allowed to depend upon). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2776) Master should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577701#comment-14577701 ] Yan Xu edited comment on MESOS-2776 at 6/8/15 8:19 PM: --- https://reviews.apache.org/r/35118/ https://reviews.apache.org/r/35119/ was (Author: vinodkone): https://reviews.apache.org/r/35119/ > Master should expose metrics about oversubscribed resources > --- > > Key: MESOS-2776 > URL: https://issues.apache.org/jira/browse/MESOS-2776 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Yan Xu > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2743) Include ExecutorInfos in master/state.json
[ https://issues.apache.org/jira/browse/MESOS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2743: --- Story Points: 3 > Include ExecutorInfos in master/state.json > -- > > Key: MESOS-2743 > URL: https://issues.apache.org/jira/browse/MESOS-2743 > Project: Mesos > Issue Type: Improvement > Components: json api >Reporter: Adam B >Assignee: haosdent > Labels: mesosphere > > The slave/state.json already reports executorInfos: > https://github.com/apache/mesos/blob/0.22.1/src/slave/http.cpp#L215-219 > Would be great to see this in the master/state.json as well, so external > tools don't have to query each slave to find out executor resources, sandbox > directories, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2673) Follow Google Style Guide for header file include order completely.
[ https://issues.apache.org/jira/browse/MESOS-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2673: --- Sprint: Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint 10, Mesosphere Sprint 11) > Follow Google Style Guide for header file include order completely. > --- > > Key: MESOS-2673 > URL: https://issues.apache.org/jira/browse/MESOS-2673 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad >Priority: Minor > Fix For: 0.23.0 > > > The header include order for Mesos actually follows the Google Styleguide but > omits step 1 without mentioning this exception in the Mesos styleguide. This > proposal suggests to adapt to the include order explained in the Google > Styleguide i.e. include the direct headers first in the .cpp files > implementing them. > A gist of the proposal can be found here: > https://gist.github.com/joerg84/65cb9611d24b2e35b69b > The corresponding Review Board review can be found here: > https://reviews.apache.org/r/33646/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2807) As a developer I need an easy way to convert MasterInfo protobuf to/from JSON
[
https://issues.apache.org/jira/browse/MESOS-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2807:
---
Sprint: Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint
11)
> As a developer I need an easy way to convert MasterInfo protobuf to/from JSON
> -
>
> Key: MESOS-2807
> URL: https://issues.apache.org/jira/browse/MESOS-2807
> Project: Mesos
> Issue Type: Task
> Components: leader election
>Reporter: Marco Massenzio
>Assignee: Marco Massenzio
> Fix For: 0.23.0
>
>
> As a preliminary to MESOS-2340, this requires the implementation of a simple
> (de)serialization mechanism to JSON from/to {{MasterInfo}} protobuf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2805) Make synchronized as primary form of synchronization.
[
https://issues.apache.org/jira/browse/MESOS-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2805:
---
Sprint: Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint
11)
> Make synchronized as primary form of synchronization.
> -
>
> Key: MESOS-2805
> URL: https://issues.apache.org/jira/browse/MESOS-2805
> Project: Mesos
> Issue Type: Improvement
> Components: libprocess, stout
>Reporter: Joris Van Remoortere
>Assignee: Joris Van Remoortere
> Labels: c++11, libprocess, stout
>
> Re-organize Synchronized to allow {{synchronized(m)}} to work on:
> 1. {{std::mutex}}
> 2. {{std::recursive_mutex}}
> 3. {{std::atomic_flag}}
> Move synchronized.hpp into stout, so that developers don't think it's part of
> the utility suite for actors in libprocess.
> Remove references to internal.hpp and replace them with {{std::atomic_flag}}
> synchronization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1790) Add "chown" option to CommandInfo.URI
[ https://issues.apache.org/jira/browse/MESOS-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-1790: --- Sprint: Mesosphere Sprint 12 > Add "chown" option to CommandInfo.URI > - > > Key: MESOS-1790 > URL: https://issues.apache.org/jira/browse/MESOS-1790 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Jim Klucar > Labels: mesosphere, newbie > Attachments: > 0001-MESOS-1790-Adds-chown-option-to-CommandInfo.URI.patch > > > Mesos fetcher always chown()s the extracted executor URIs as the executor > user but sometimes this is not desirable, e.g., "setuid" bit gets lost during > chown() if slave/fetcher is running as root. > It would be nice to give frameworks the ability to skip the chown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2801) Remove dynamic allocation from Future
[ https://issues.apache.org/jira/browse/MESOS-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2801: --- Sprint: Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint 11) > Remove dynamic allocation from Future > > > Key: MESOS-2801 > URL: https://issues.apache.org/jira/browse/MESOS-2801 > Project: Mesos > Issue Type: Improvement > Components: libprocess >Reporter: Joris Van Remoortere >Assignee: Joris Van Remoortere > > Remove the dynamic allocation of `T*` inside `Future::Data` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2653) Slave should act on correction events from QoS controller
[
https://issues.apache.org/jira/browse/MESOS-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-2653:
---
Sprint: Mesosphere Sprint 12
> Slave should act on correction events from QoS controller
> -
>
> Key: MESOS-2653
> URL: https://issues.apache.org/jira/browse/MESOS-2653
> Project: Mesos
> Issue Type: Improvement
>Reporter: Vinod Kone
>Assignee: Niklas Quarfot Nielsen
> Labels: mesosphere
>
> Slave might want to kill revocable tasks based on correction events from the
> QoS controller.
> The QoS controller communicates corrections through a stream (or
> process::Queue) to the slave which corrections it needs to carry out, in
> order to mitigate interference with production tasks.
> The correction is communicated through a message:
> [code]
> message QoSCorrection {
> enum CorrectionType {
> KillExecutor = 1
> // KillTask = 2
> // Resize, throttle task
> }
> optional string reason = X;
> optional ExecutorID executor_id = X;
> // optional TaskID task_id = X;
> }
> [/code]
> And the slave will setup a handler to process these events. Initially, only
> executor termination is supported and cause the slave to issue
> 'containerizer->destroy()'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2704) Add tests for QoS controller corrections
[ https://issues.apache.org/jira/browse/MESOS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2704: --- Sprint: Mesosphere Sprint 12 > Add tests for QoS controller corrections > > > Key: MESOS-2704 > URL: https://issues.apache.org/jira/browse/MESOS-2704 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Niklas Quarfot Nielsen > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo
[ https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2340: --- Sprint: Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Sprint 11) > Publish JSON in ZK instead of serialized MasterInfo > --- > > Key: MESOS-2340 > URL: https://issues.apache.org/jira/browse/MESOS-2340 > Project: Mesos > Issue Type: Improvement > Components: leader election >Reporter: Zameer Manji >Assignee: Marco Massenzio > > Currently to discover the master a client needs the ZK node location and > access to the MasterInfo protobuf so it can deserialize the binary blob in > the node. > I think it would be nice to publish JSON (like Twitter's ServerSets) so > clients are not tied to protobuf to do service discovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2205) Add user documentation for reservations
[ https://issues.apache.org/jira/browse/MESOS-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2205: --- Sprint: Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11) > Add user documentation for reservations > --- > > Key: MESOS-2205 > URL: https://issues.apache.org/jira/browse/MESOS-2205 > Project: Mesos > Issue Type: Documentation > Components: documentation, framework >Reporter: Michael Park >Assignee: Michael Park > Labels: mesosphere > > Add a user guide for reservations which describes basic usage of them, how > ACLs are used to specify who can unreserve whose resources, and few advanced > usage cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2085) Add support encrypted and non-encrypted communication in parallel for cluster upgrade
[ https://issues.apache.org/jira/browse/MESOS-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2085: --- Sprint: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11) > Add support encrypted and non-encrypted communication in parallel for cluster > upgrade > - > > Key: MESOS-2085 > URL: https://issues.apache.org/jira/browse/MESOS-2085 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > > During cluster upgrade from non-encrypted to encrypted communication, we need > to support an interim where: > 1) A master can have connections to both encrypted and non-encrypted slaves > 2) A slave that supports encrypted communication connects to a master that > has not yet been upgraded. > 3) Frameworks are encrypted but the master has not been upgraded yet. > 4) Master has been upgraded but frameworks haven't. > 5) A slave process has upgraded but running executor processes haven't. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2110) Configurable Ping Timeouts
[ https://issues.apache.org/jira/browse/MESOS-2110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2110: --- Sprint: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11) > Configurable Ping Timeouts > -- > > Key: MESOS-2110 > URL: https://issues.apache.org/jira/browse/MESOS-2110 > Project: Mesos > Issue Type: Improvement > Components: master, slave >Reporter: Adam B >Assignee: Adam B > Labels: master, network, slave, timeout > > After a series of ping-failures, the master considers the slave lost and > calls shutdownSlave, requiring such a slave that reconnects to kill its tasks > and re-register as a new slaveId. On the other side, after a similar timeout, > the slave will consider the master lost and try to detect a new master. These > timeouts are currently hardcoded constants (5 * 15s), which may not be > well-suited for all scenarios. > - Some clusters may tolerate a longer slave process restart period, and > wouldn't want tasks to be killed upon reconnect. > - Some clusters may have higher-latency networks (e.g. cross-datacenter, or > for volunteer computing efforts), and would like to tolerate longer periods > without communication. > We should provide flags/mechanisms on the master to control its tolerance for > non-communicative slaves, and (less importantly?) on the slave to tolerate > missing masters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1913) Create libevent/SSL-backed Socket implementation
[ https://issues.apache.org/jira/browse/MESOS-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-1913: --- Sprint: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11) > Create libevent/SSL-backed Socket implementation > > > Key: MESOS-1913 > URL: https://issues.apache.org/jira/browse/MESOS-1913 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2108) Add configure flag or environment variable to enable SSL/libevent Socket
[ https://issues.apache.org/jira/browse/MESOS-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2108: --- Sprint: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11, Mesosphere Sprint 12 (was: Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20, Mesosphere Q1 Sprint 4 - 3/6, Mesosphere Q1 Sprint 5 - 3/20, Mesosphere Q1 Sprint 6 - 4/3, Mesosphere Q1 Sprint 7 - 4/17, Mesosphere Q2 Sprint 8 - 5/1, Mesosphere Q1 Sprint 9 - 5/15, Mesosphere Sprint 10, Mesosphere Sprint 11) > Add configure flag or environment variable to enable SSL/libevent Socket > > > Key: MESOS-2108 > URL: https://issues.apache.org/jira/browse/MESOS-2108 > Project: Mesos > Issue Type: Task >Reporter: Niklas Quarfot Nielsen >Assignee: Joris Van Remoortere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2821) Document and consolidate qdisc handles
[ https://issues.apache.org/jira/browse/MESOS-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577750#comment-14577750 ] Paul Brett commented on MESOS-2821: --- Review completed and changes applied. > Document and consolidate qdisc handles > -- > > Key: MESOS-2821 > URL: https://issues.apache.org/jira/browse/MESOS-2821 > Project: Mesos > Issue Type: Improvement >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > The structure of traffic control qdiscs and filters in non-trivial with the > knowledge of which handles are the parents of which filters or qdiscs are in > the create and recovery functions and will be needed to collect statistics on > the links. Lets pull out the constants and document them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2829) FetcherCacheTest.CachedFallback test is flaky
[
https://issues.apache.org/jira/browse/MESOS-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-2829:
--
Component/s: fetcher
> FetcherCacheTest.CachedFallback test is flaky
> -
>
> Key: MESOS-2829
> URL: https://issues.apache.org/jira/browse/MESOS-2829
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 0.23.0
>Reporter: Vinod Kone
>Assignee: Bernd Mathiske
> Labels: flaky-test
>
> Observed this on internal CI
> {code}
> DEBUG: [ RUN ] FetcherCacheTest.CachedFallback
> DEBUG: Using temporary directory '/tmp/FetcherCacheTest_CachedFallback_dmr8bH'
> DEBUG: I0608 08:32:39.091538 56580 leveldb.cpp:176] Opened db in 5.078136ms
> DEBUG: I0608 08:32:39.093271 56580 leveldb.cpp:183] Compacted db in 1.696938ms
> DEBUG: I0608 08:32:39.093312 56580 leveldb.cpp:198] Created db iterator in
> 5708ns
> DEBUG: I0608 08:32:39.093325 56580 leveldb.cpp:204] Seeked to beginning of db
> in 940ns
> DEBUG: I0608 08:32:39.09 56580 leveldb.cpp:273] Iterated through 0 keys
> in the db in 270ns
> DEBUG: I0608 08:32:39.093353 56580 replica.cpp:744] Replica recovered with
> log positions 0 -> 0 with 1 holes and 0 unlearned
> DEBUG: I0608 08:32:39.093597 56613 recover.cpp:449] Starting replica recovery
> DEBUG: I0608 08:32:39.093706 56597 recover.cpp:475] Replica is in EMPTY status
> DEBUG: I0608 08:32:39.094511 56607 replica.cpp:641] Replica in EMPTY status
> received a broadcasted recover request
> DEBUG: I0608 08:32:39.094952 56606 recover.cpp:195] Received a recover
> response from a replica in EMPTY status
> DEBUG: I0608 08:32:39.095067 56606 recover.cpp:566] Updating replica status
> to STARTING
> DEBUG: I0608 08:32:39.095487 56598 master.cpp:363] Master
> 20150608-083239-1787367596-39187-56580 (<***redacted***>) started on
> <***redacted***>
> DEBUG: I0608 08:32:39.095510 56598 master.cpp:365] Flags at startup:
> --acls="" --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5"
> --credentials="/tmp/FetcherCacheTest_CachedFallback_dmr8bH/credentials"
> --framework_sorter="drf" --help="false" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --quiet="false" --recovery_slave_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_store_timeout="25secs" --registry_strict="true"
> --root_submissions="true" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false"
> --webui_dir="/usr/local/share/mesos/webui"
> --work_dir="/tmp/FetcherCacheTest_CachedFallback_dmr8bH/master"
> --zk_session_timeout="10secs"
> DEBUG: I0608 08:32:39.095630 56598 master.cpp:410] Master only allowing
> authenticated frameworks to register
> DEBUG: I0608 08:32:39.095636 56598 master.cpp:415] Master only allowing
> authenticated slaves to register
> DEBUG: I0608 08:32:39.095643 56598 credentials.hpp:37] Loading credentials
> for authentication from
> '/tmp/FetcherCacheTest_CachedFallback_dmr8bH/credentials'
> DEBUG: I0608 08:32:39.095743 56598 master.cpp:454] Using default 'crammd5'
> authenticator
> DEBUG: I0608 08:32:39.095782 56598 master.cpp:491] Authorization enabled
> DEBUG: I0608 08:32:39.096765 56606 leveldb.cpp:306] Persisting metadata (8
> bytes) to leveldb took 1.654474ms
> DEBUG: I0608 08:32:39.096788 56606 replica.cpp:323] Persisted replica status
> to STARTING
> DEBUG: I0608 08:32:39.096925 56612 master.cpp:1474] The newly elected leader
> is master@<***redacted***> with id 20150608-083239-1787367596-39187-56580
> DEBUG: I0608 08:32:39.096940 56612 master.cpp:1487] Elected as the leading
> master!
> DEBUG: I0608 08:32:39.096946 56612 master.cpp:1257] Recovering from registrar
> DEBUG: I0608 08:32:39.097033 56608 registrar.cpp:313] Recovering registrar
> DEBUG: I0608 08:32:39.097095 56616 recover.cpp:475] Replica is in STARTING
> status
> DEBUG: I0608 08:32:39.097718 56607 replica.cpp:641] Replica in STARTING
> status received a broadcasted recover request
> DEBUG: I0608 08:32:39.097949 56602 recover.cpp:195] Received a recover
> response from a replica in STARTING status
> DEBUG: I0608 08:32:39.098423 56609 r
[jira] [Updated] (MESOS-2831) FetcherCacheTest.SimpleEviction is flaky
[
https://issues.apache.org/jira/browse/MESOS-2831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-2831:
--
Component/s: fetcher
> FetcherCacheTest.SimpleEviction is flaky
>
>
> Key: MESOS-2831
> URL: https://issues.apache.org/jira/browse/MESOS-2831
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 0.23.0
>Reporter: Vinod Kone
>Assignee: Bernd Mathiske
> Labels: flaky-test
>
> Saw this when reviewbot was testing an unrelated review
> https://reviews.apache.org/r/35119/
> {code}
> [ RUN ] FetcherCacheTest.SimpleEviction
> GMOCK WARNING:
> Uninteresting mock function call - returning directly.
> Function call: resourceOffers(0x5365320, @0x2b7bef9f1b20 { 128-byte
> object 00-00 C0-75 00-18 7C-2B 00-00 60-76 00-18 7C-2B 00-00 00-77 00-18 7C-2B 00-00
> 40-3A 00-18 7C-2B 00-00 04-00 00-00 04-00 00-00 04-00 00-00 7C-2B 00-00 00-00
> 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00
> 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 0F-00
> 00-00> })
> Stack trace:
> F0607 21:19:23.181392 4246 fetcher_cache_tests.cpp:354] CHECK_READY(offers):
> is PENDING Failed to wait for resource offers
> *** Check failure stack trace: ***
> @ 0x2b7be56c5972 google::LogMessage::Fail()
> @ 0x2b7be56c58be google::LogMessage::SendToLog()
> @ 0x2b7be56c52c0 google::LogMessage::Flush()
> @ 0x2b7be56c81d4 google::LogMessageFatal::~LogMessageFatal()
> @ 0x97d182 _CheckFatal::~_CheckFatal()
> @ 0xb58a28
> mesos::internal::tests::FetcherCacheTest::launchTask()
> @ 0xb65b50
> mesos::internal::tests::FetcherCacheTest_SimpleEviction_Test::TestBody()
> @ 0x11923b7
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @ 0x118d5b4
> testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x1175975 testing::Test::Run()
> @ 0x1176098 testing::TestInfo::Run()
> @ 0x1176620 testing::TestCase::Run()
> @ 0x117b2ea testing::internal::UnitTestImpl::RunAllTests()
> @ 0x1193229
> testing::internal::HandleSehExceptionsInMethodIfSupported<>()
> @ 0x118e2a5
> testing::internal::HandleExceptionsInMethodIfSupported<>()
> @ 0x117a1f6 testing::UnitTest::Run()
> @ 0xcc832b main
> @ 0x2b7be7d46ec5 (unknown)
> @ 0x872379 (unknown)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-2831) FetcherCacheTest.SimpleEviction is flaky
Vinod Kone created MESOS-2831:
-
Summary: FetcherCacheTest.SimpleEviction is flaky
Key: MESOS-2831
URL: https://issues.apache.org/jira/browse/MESOS-2831
Project: Mesos
Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Vinod Kone
Assignee: Bernd Mathiske
Saw this when reviewbot was testing an unrelated review
https://reviews.apache.org/r/35119/
{code}
[ RUN ] FetcherCacheTest.SimpleEviction
GMOCK WARNING:
Uninteresting mock function call - returning directly.
Function call: resourceOffers(0x5365320, @0x2b7bef9f1b20 { 128-byte object
})
Stack trace:
F0607 21:19:23.181392 4246 fetcher_cache_tests.cpp:354] CHECK_READY(offers):
is PENDING Failed to wait for resource offers
*** Check failure stack trace: ***
@ 0x2b7be56c5972 google::LogMessage::Fail()
@ 0x2b7be56c58be google::LogMessage::SendToLog()
@ 0x2b7be56c52c0 google::LogMessage::Flush()
@ 0x2b7be56c81d4 google::LogMessageFatal::~LogMessageFatal()
@ 0x97d182 _CheckFatal::~_CheckFatal()
@ 0xb58a28 mesos::internal::tests::FetcherCacheTest::launchTask()
@ 0xb65b50
mesos::internal::tests::FetcherCacheTest_SimpleEviction_Test::TestBody()
@ 0x11923b7
testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@ 0x118d5b4
testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x1175975 testing::Test::Run()
@ 0x1176098 testing::TestInfo::Run()
@ 0x1176620 testing::TestCase::Run()
@ 0x117b2ea testing::internal::UnitTestImpl::RunAllTests()
@ 0x1193229
testing::internal::HandleSehExceptionsInMethodIfSupported<>()
@ 0x118e2a5
testing::internal::HandleExceptionsInMethodIfSupported<>()
@ 0x117a1f6 testing::UnitTest::Run()
@ 0xcc832b main
@ 0x2b7be7d46ec5 (unknown)
@ 0x872379 (unknown)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2776) Master should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577701#comment-14577701 ] Vinod Kone commented on MESOS-2776: --- https://reviews.apache.org/r/35119/ > Master should expose metrics about oversubscribed resources > --- > > Key: MESOS-2776 > URL: https://issues.apache.org/jira/browse/MESOS-2776 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Yan Xu > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2804) Log framework capabilities in the master.
[
https://issues.apache.org/jira/browse/MESOS-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-2804:
--
Assignee: Benjamin Mahler
> Log framework capabilities in the master.
> -
>
> Key: MESOS-2804
> URL: https://issues.apache.org/jira/browse/MESOS-2804
> Project: Mesos
> Issue Type: Improvement
> Components: master
>Reporter: Benjamin Mahler
>Assignee: Benjamin Mahler
>Priority: Minor
> Labels: twitter
>
> Now that {{Capabilities}} has been added to FrameworkInfo, we should log
> these in the master when a framework (re-)registers (i.e. which capabilities
> are enabled and disabled). This would make debugging easier for framework
> developers.
> Ideally, folding in the old {{checkpoint}} capability and logging that as
> well. In the past, the fact that {{checkpoint}} defaults to false has tripped
> up a lot of developers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2804) Log framework capabilities in the master.
[
https://issues.apache.org/jira/browse/MESOS-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-2804:
--
Sprint: Twitter Mesos Q2 Sprint 5
Story Points: 1
> Log framework capabilities in the master.
> -
>
> Key: MESOS-2804
> URL: https://issues.apache.org/jira/browse/MESOS-2804
> Project: Mesos
> Issue Type: Improvement
> Components: master
>Reporter: Benjamin Mahler
>Priority: Minor
> Labels: twitter
>
> Now that {{Capabilities}} has been added to FrameworkInfo, we should log
> these in the master when a framework (re-)registers (i.e. which capabilities
> are enabled and disabled). This would make debugging easier for framework
> developers.
> Ideally, folding in the old {{checkpoint}} capability and logging that as
> well. In the past, the fact that {{checkpoint}} defaults to false has tripped
> up a lot of developers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2775) Slave should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2775: -- Assignee: Benjamin Mahler (was: Vinod Kone) > Slave should expose metrics about oversubscribed resources > -- > > Key: MESOS-2775 > URL: https://issues.apache.org/jira/browse/MESOS-2775 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Benjamin Mahler > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2830) Add an endpoint to slaves to allow launching system administration tasks
Cody Maloney created MESOS-2830: --- Summary: Add an endpoint to slaves to allow launching system administration tasks Key: MESOS-2830 URL: https://issues.apache.org/jira/browse/MESOS-2830 Project: Mesos Issue Type: Wish Components: slave Reporter: Cody Maloney Priority: Minor As a System Administrator often times I need to run a organization-mandated task on every machine in the cluster. Ideally I could do this within the framework of mesos resources if it is a "cleanup" or auditing task, but sometimes I just have to run something, and run it now, regardless if a machine has un-accounted resources (Ex: Adding/removing a user). Currently to do this I have to completely bypass Mesos and SSH to the box. Ideally I could tell a mesos slave (With proper authentication) to run a container with the limited special permissions needed to get the task done. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2818) Pass 'allocated' resources for each executor to the resource estimator.
[
https://issues.apache.org/jira/browse/MESOS-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577659#comment-14577659
]
Jie Yu commented on MESOS-2818:
---
Here is a new proposal. I personally think that's better than all existing
proposals. Here is the summary.
1) We kill the ResourceMonitor and move its logic (currently, only 'usages()')
into the slave
{code}
Future Slave::usage() { ... }
message ResourceUsage {
message Executor {
optional ExecutorInfo executor_info = 1;
repeated Resource allocated = 2;
repeated ResourceStatistics statistics = 3;
}
repeated Resource total = 1; // Slave's total resources.
repeated Executor executors = 2; // Per-executor allocated/usage information.
}
{code}
2) Slave collect per-executor allocated information when the resource estimator
calls 'usage()':
{code}
Future Slave::usage()
{
ResourceUsage usage;
list futures;
foreachvalue (const Framework* framework, frameworks) {
foreachvalue (const Executor* executor, framework->executors) {
ResourceUsage::Executor* entry = usage.add_executors()'
entry->set_allocated(...);
futures.push_back(containerizer->usage(executor->containerId));
}
}
return await(futures)
.then(defer(self(), &Self::_usage, usage, lambda::_1));
}
{code}
3) In this way, it's guaranteed that if an executor has 'statistics', it will
have 'allocated' info available. Vice not versa. But in that case, we can just
make conservative estimation (i.e., assuming all resources allocated are
consumed).
> Pass 'allocated' resources for each executor to the resource estimator.
> ---
>
> Key: MESOS-2818
> URL: https://issues.apache.org/jira/browse/MESOS-2818
> Project: Mesos
> Issue Type: Task
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Resource estimator obviously need this information to calculate, say the
> usage slack. Now the question is how. There are two approaches:
> 1) Pass in the allocated resources for each executor through the
> 'oversubscribable()' interface.
> 2) Let containerizer return total resources allocated for each container when
> 'usages()' are invoked.
> I would suggest to take route (1) for several reasons:
> 1) Eventually, we'll need to pass in slave's total resources to the resource
> estimator (so that RE can calculate allocation slack). There is no way that
> we can get that from containerizer. The slave's total resources keep changing
> due to dynamic reservation. So we cannot pass in the slave total resources
> during initialization.
> 2) The current implementation of usages() might skip some containers if it
> fails to get statistics for that container (not an error). This will cause
> in-complete information to the RE.
> 3) We may want to calculate 'unallocated = total - allocated' so that we can
> send allocation slack as well. Getting 'total' and 'allocated' from two
> different components might result in inconsistent value. Remember that
> 'total' keeps changing due to dynamic reservation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-1104) TODO(benh): Move linux/fs.hpp out of 'mesos- namespace in linux/fs.h
[ https://issues.apache.org/jira/browse/MESOS-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jojy Varghese reassigned MESOS-1104: Assignee: Jojy Varghese > TODO(benh): Move linux/fs.hpp out of 'mesos- namespace in linux/fs.h > > > Key: MESOS-1104 > URL: https://issues.apache.org/jira/browse/MESOS-1104 > Project: Mesos > Issue Type: Improvement >Reporter: Archana kumari >Assignee: Jojy Varghese > Labels: newbie > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2823) Pass callback to the QoS Controller to retrieve ResourceUsage from Resource Monitor on demand.
[ https://issues.apache.org/jira/browse/MESOS-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen updated MESOS-2823: -- Story Points: 2 > Pass callback to the QoS Controller to retrieve ResourceUsage from Resource > Monitor on demand. > -- > > Key: MESOS-2823 > URL: https://issues.apache.org/jira/browse/MESOS-2823 > Project: Mesos > Issue Type: Task >Reporter: Bartek Plotka >Assignee: Bartek Plotka > Labels: intel > > We need to allow QoS Controller to call 'ResourceMonitor::usages()'. We will > pass it in a lambda. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code
[ https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen updated MESOS-2637: -- Story Points: 2 > Consolidate 'foo', 'bar', ... string constants in test and example code > --- > > Key: MESOS-2637 > URL: https://issues.apache.org/jira/browse/MESOS-2637 > Project: Mesos > Issue Type: Bug > Components: technical debt >Reporter: Niklas Quarfot Nielsen >Assignee: Colin Williams > > We are using 'foo', 'bar', ... string constants and pairs in > src/tests/master_tests.cpp, src/tests/slave_tests.cpp, > src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and > hooks tests. We should consolidate them to make the call sites less prone to > forgetting to update all call sites. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo
[ https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen updated MESOS-2340: -- Shepherd: (was: Niklas Quarfot Nielsen) > Publish JSON in ZK instead of serialized MasterInfo > --- > > Key: MESOS-2340 > URL: https://issues.apache.org/jira/browse/MESOS-2340 > Project: Mesos > Issue Type: Improvement > Components: leader election >Reporter: Zameer Manji >Assignee: Marco Massenzio > > Currently to discover the master a client needs the ZK node location and > access to the MasterInfo protobuf so it can deserialize the binary blob in > the node. > I think it would be nice to publish JSON (like Twitter's ServerSets) so > clients are not tied to protobuf to do service discovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2818) Pass 'allocated' resources for each executor to the resource estimator.
[ https://issues.apache.org/jira/browse/MESOS-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577632#comment-14577632 ] Jie Yu edited comment on MESOS-2818 at 6/8/15 6:44 PM: --- Chatted this with BenM and Vinod, there is still a race condition that this solution cannot prevent. Here is the timeline: 1) Slave wants to call estimator->oversubscribable(..) 2) Slave calculates 'allocated' and pass it to 'oversubscribable(..)' 3) A new task T gets launched in the slave 4) Resource estimator calls 'usages()' 5) The results from 'usages()' has information about T while T does not exist in 'allocated' 6) The resource estimator does not know how to calculate usage slack was (Author: jieyu): Chatted this with BenM and Vinod, there is still a race condition that this solution cannot prevent. There is the timeline: 1) Slave wants to call estimator->oversubscribable(..) 2) Slave calculates 'allocated' and pass it to 'oversubscribable(..)' 3) A new task T gets launched in the slave 4) Resource estimator calls 'usages()' 5) The results from 'usages()' has information about T while T does not exist in 'allocated' 6) The resource estimator does not know how to calculate usage slack > Pass 'allocated' resources for each executor to the resource estimator. > --- > > Key: MESOS-2818 > URL: https://issues.apache.org/jira/browse/MESOS-2818 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jie Yu > > Resource estimator obviously need this information to calculate, say the > usage slack. Now the question is how. There are two approaches: > 1) Pass in the allocated resources for each executor through the > 'oversubscribable()' interface. > 2) Let containerizer return total resources allocated for each container when > 'usages()' are invoked. > I would suggest to take route (1) for several reasons: > 1) Eventually, we'll need to pass in slave's total resources to the resource > estimator (so that RE can calculate allocation slack). There is no way that > we can get that from containerizer. The slave's total resources keep changing > due to dynamic reservation. So we cannot pass in the slave total resources > during initialization. > 2) The current implementation of usages() might skip some containers if it > fails to get statistics for that container (not an error). This will cause > in-complete information to the RE. > 3) We may want to calculate 'unallocated = total - allocated' so that we can > send allocation slack as well. Getting 'total' and 'allocated' from two > different components might result in inconsistent value. Remember that > 'total' keeps changing due to dynamic reservation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2818) Pass 'allocated' resources for each executor to the resource estimator.
[ https://issues.apache.org/jira/browse/MESOS-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577632#comment-14577632 ] Jie Yu edited comment on MESOS-2818 at 6/8/15 6:44 PM: --- Chatted this with BenM and Vinod, there is still a race condition that this solution cannot prevent. Here is the timeline: 1) Slave wants to call estimator->oversubscribable(..) 2) Slave calculates 'allocated' and pass it to 'oversubscribable(..)' 3) A new task T gets launched in the slave 4) Resource estimator calls 'usages()' 5) The results from 'usages()' has information about T while T does not exist in 'allocated' 6) The resource estimator does not know how to calculate usage slack for T was (Author: jieyu): Chatted this with BenM and Vinod, there is still a race condition that this solution cannot prevent. Here is the timeline: 1) Slave wants to call estimator->oversubscribable(..) 2) Slave calculates 'allocated' and pass it to 'oversubscribable(..)' 3) A new task T gets launched in the slave 4) Resource estimator calls 'usages()' 5) The results from 'usages()' has information about T while T does not exist in 'allocated' 6) The resource estimator does not know how to calculate usage slack > Pass 'allocated' resources for each executor to the resource estimator. > --- > > Key: MESOS-2818 > URL: https://issues.apache.org/jira/browse/MESOS-2818 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jie Yu > > Resource estimator obviously need this information to calculate, say the > usage slack. Now the question is how. There are two approaches: > 1) Pass in the allocated resources for each executor through the > 'oversubscribable()' interface. > 2) Let containerizer return total resources allocated for each container when > 'usages()' are invoked. > I would suggest to take route (1) for several reasons: > 1) Eventually, we'll need to pass in slave's total resources to the resource > estimator (so that RE can calculate allocation slack). There is no way that > we can get that from containerizer. The slave's total resources keep changing > due to dynamic reservation. So we cannot pass in the slave total resources > during initialization. > 2) The current implementation of usages() might skip some containers if it > fails to get statistics for that container (not an error). This will cause > in-complete information to the RE. > 3) We may want to calculate 'unallocated = total - allocated' so that we can > send allocation slack as well. Getting 'total' and 'allocated' from two > different components might result in inconsistent value. Remember that > 'total' keeps changing due to dynamic reservation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2818) Pass 'allocated' resources for each executor to the resource estimator.
[ https://issues.apache.org/jira/browse/MESOS-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577632#comment-14577632 ] Jie Yu commented on MESOS-2818: --- Chatted this with BenM and Vinod, there is still a race condition that this solution cannot prevent. There is the timeline: 1) Slave wants to call estimator->oversubscribable(..) 2) Slave calculates 'allocated' and pass it to 'oversubscribable(..)' 3) A new task T gets launched in the slave 4) Resource estimator calls 'usages()' 5) The results from 'usages()' has information about T while T does not exist in 'allocated' 6) The resource estimator does not know how to calculate usage slack > Pass 'allocated' resources for each executor to the resource estimator. > --- > > Key: MESOS-2818 > URL: https://issues.apache.org/jira/browse/MESOS-2818 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jie Yu > > Resource estimator obviously need this information to calculate, say the > usage slack. Now the question is how. There are two approaches: > 1) Pass in the allocated resources for each executor through the > 'oversubscribable()' interface. > 2) Let containerizer return total resources allocated for each container when > 'usages()' are invoked. > I would suggest to take route (1) for several reasons: > 1) Eventually, we'll need to pass in slave's total resources to the resource > estimator (so that RE can calculate allocation slack). There is no way that > we can get that from containerizer. The slave's total resources keep changing > due to dynamic reservation. So we cannot pass in the slave total resources > during initialization. > 2) The current implementation of usages() might skip some containers if it > fails to get statistics for that container (not an error). This will cause > in-complete information to the RE. > 3) We may want to calculate 'unallocated = total - allocated' so that we can > send allocation slack as well. Getting 'total' and 'allocated' from two > different components might result in inconsistent value. Remember that > 'total' keeps changing due to dynamic reservation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2775) Slave should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone reassigned MESOS-2775: - Assignee: Vinod Kone (was: brian wickman) > Slave should expose metrics about oversubscribed resources > -- > > Key: MESOS-2775 > URL: https://issues.apache.org/jira/browse/MESOS-2775 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2775) Slave should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2775: -- Sprint: Twitter Q2 Sprint 4 > Slave should expose metrics about oversubscribed resources > -- > > Key: MESOS-2775 > URL: https://issues.apache.org/jira/browse/MESOS-2775 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2776) Master should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-2776: -- Labels: twitter (was: ) > Master should expose metrics about oversubscribed resources > --- > > Key: MESOS-2776 > URL: https://issues.apache.org/jira/browse/MESOS-2776 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Yan Xu > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2776) Master should expose metrics about oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2776: -- Sprint: Twitter Q2 Sprint 4 > Master should expose metrics about oversubscribed resources > --- > > Key: MESOS-2776 > URL: https://issues.apache.org/jira/browse/MESOS-2776 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Yan Xu > Labels: twitter > > state.json and metrics/snapshot should expose metrics on oversubscribed > resources (allocated and available). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-2817) Support revocable/non-revocable CPU updates in Mesos containerizer
[ https://issues.apache.org/jira/browse/MESOS-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Downes resolved MESOS-2817. --- Resolution: Won't Fix Consensus is to only support a single resource being entirely revocable/non-revocable so the behavior is intuitive and therefore the existing code for the CPU isolator is sufficient. > Support revocable/non-revocable CPU updates in Mesos containerizer > -- > > Key: MESOS-2817 > URL: https://issues.apache.org/jira/browse/MESOS-2817 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.23.0 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > MESOS-2652 provided preliminary support for revocable cpu resources only when > specified in the initial resources for a container. Improve this to support > updates to/from revocable cpu. Note, *any* revocable cpu will result in the > entire container's cpu being treated as revocable at the cpu isolator level. > Higher level logic is responsible for adding/removing based on some policy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2791) Create a FixedResourceEstimator to return fixed amount of oversubscribable resources.
[ https://issues.apache.org/jira/browse/MESOS-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2791: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Create a FixedResourceEstimator to return fixed amount of oversubscribable > resources. > - > > Key: MESOS-2791 > URL: https://issues.apache.org/jira/browse/MESOS-2791 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jie Yu > > This will be useful for testing oversubscription in a real environment. Also, > it will be useful for people who has a prior knowledge about the amount of > resources that can be safely oversubscribed on each slave. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2818) Pass 'allocated' resources for each executor to the resource estimator.
[ https://issues.apache.org/jira/browse/MESOS-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2818: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Pass 'allocated' resources for each executor to the resource estimator. > --- > > Key: MESOS-2818 > URL: https://issues.apache.org/jira/browse/MESOS-2818 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu >Assignee: Jie Yu > > Resource estimator obviously need this information to calculate, say the > usage slack. Now the question is how. There are two approaches: > 1) Pass in the allocated resources for each executor through the > 'oversubscribable()' interface. > 2) Let containerizer return total resources allocated for each container when > 'usages()' are invoked. > I would suggest to take route (1) for several reasons: > 1) Eventually, we'll need to pass in slave's total resources to the resource > estimator (so that RE can calculate allocation slack). There is no way that > we can get that from containerizer. The slave's total resources keep changing > due to dynamic reservation. So we cannot pass in the slave total resources > during initialization. > 2) The current implementation of usages() might skip some containers if it > fails to get statistics for that container (not an error). This will cause > in-complete information to the RE. > 3) We may want to calculate 'unallocated = total - allocated' so that we can > send allocation slack as well. Getting 'total' and 'allocated' from two > different components might result in inconsistent value. Remember that > 'total' keeps changing due to dynamic reservation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2821) Document and consolidate qdisc handles
[ https://issues.apache.org/jira/browse/MESOS-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2821: -- Sprint: Twitter Q2 Sprint 4 > Document and consolidate qdisc handles > -- > > Key: MESOS-2821 > URL: https://issues.apache.org/jira/browse/MESOS-2821 > Project: Mesos > Issue Type: Improvement >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > The structure of traffic control qdiscs and filters in non-trivial with the > knowledge of which handles are the parents of which filters or qdiscs are in > the create and recovery functions and will be needed to collect statistics on > the links. Lets pull out the constants and document them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2752) Add HTB queueing discipline wrapper class
[ https://issues.apache.org/jira/browse/MESOS-2752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2752: -- Sprint: Twitter Q2 Sprint 4 > Add HTB queueing discipline wrapper class > - > > Key: MESOS-2752 > URL: https://issues.apache.org/jira/browse/MESOS-2752 > Project: Mesos > Issue Type: Improvement >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > Network isolator uses a Hierarchical Token Bucket (HTB) traffic control > discipline on the egress filter inside each container as the root for adding > traffic filters. A HTB wrapper is needed to access the network statistics > for this interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2784) Add constexpr to C++11 whitelist
[ https://issues.apache.org/jira/browse/MESOS-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2784: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Add constexpr to C++11 whitelist > > > Key: MESOS-2784 > URL: https://issues.apache.org/jira/browse/MESOS-2784 > Project: Mesos > Issue Type: Improvement > Components: documentation >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > constexpr is currently used to eliminate initialization dependency issues for > non-POD objects. We should add it to the whitelist of acceptable c++11 > features in the style guide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2753) Master should validate tasks using oversubscribed resources
[ https://issues.apache.org/jira/browse/MESOS-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2753: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Master should validate tasks using oversubscribed resources > --- > > Key: MESOS-2753 > URL: https://issues.apache.org/jira/browse/MESOS-2753 > Project: Mesos > Issue Type: Task > Components: isolation, master >Affects Versions: 0.23.0 >Reporter: Ian Downes >Assignee: Vinod Kone > Labels: twitter > > Current implementation out for [review|https://reviews.apache.org/r/34310] > only supports setting the priority of containers with revocable CPU if it's > specified in the initial executor info resources. This should be enforced at > the master. > Also master should make sure that oversubscribed resources used by the task > are valid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2824) Support pre-fetching default container image on slave startup
[ https://issues.apache.org/jira/browse/MESOS-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2824: -- Sprint: Twitter Q2 Sprint 4 Assignee: Yan Xu Story Points: 5 > Support pre-fetching default container image on slave startup > - > > Key: MESOS-2824 > URL: https://issues.apache.org/jira/browse/MESOS-2824 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.23.0 >Reporter: Ian Downes >Assignee: Yan Xu >Priority: Minor > Labels: twitter > > Default container images can be specified with the --default_container_info > flag to the slave. This may be a large image that will take a long time to > initially fetch/hash/extract when the first container is provisioned. Add > optional support to start fetching the image when the slave starts and > consider not registering until the fetch is complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2350) Add support for MesosContainerizerLaunch to chroot to a specified path
[ https://issues.apache.org/jira/browse/MESOS-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2350: -- Sprint: Twitter Mesos Q1 Sprint 3, Twitter Mesos Q1 Sprint 4, Twitter Mesos Q1 Sprint 5, Twitter Mesos Q1 Sprint 6, Twitter Q2 Sprint 1 - 4/13, Twitter Q2 Sprint 2, Twitter Q2 Sprint 3 - 5/11, Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Mesos Q1 Sprint 3, Twitter Mesos Q1 Sprint 4, Twitter Mesos Q1 Sprint 5, Twitter Mesos Q1 Sprint 6, Twitter Q2 Sprint 1 - 4/13, Twitter Q2 Sprint 2, Twitter Q2 Sprint 3 - 5/11, Twitter Q2 Sprint 3) > Add support for MesosContainerizerLaunch to chroot to a specified path > -- > > Key: MESOS-2350 > URL: https://issues.apache.org/jira/browse/MESOS-2350 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.21.1, 0.22.0 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > In preparation for the MesosContainerizer to support a filesystem isolator > the MesosContainerizerLauncher must support chrooting. Optionally, it should > also configure the chroot environment by (re-)mounting special filesystems > such as /proc and /sys and making device nodes such as /dev/zero, etc., such > that the chroot environment is functional. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2824) Support pre-fetching default container image on slave startup
[ https://issues.apache.org/jira/browse/MESOS-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Downes updated MESOS-2824: -- Labels: twitter (was: ) > Support pre-fetching default container image on slave startup > - > > Key: MESOS-2824 > URL: https://issues.apache.org/jira/browse/MESOS-2824 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.23.0 >Reporter: Ian Downes >Priority: Minor > Labels: twitter > > Default container images can be specified with the --default_container_info > flag to the slave. This may be a large image that will take a long time to > initially fetch/hash/extract when the first container is provisioned. Add > optional support to start fetching the image when the slave starts and > consider not registering until the fetch is complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2795) Introduce filesystem provisioner abstraction to Mesos containerizer
[ https://issues.apache.org/jira/browse/MESOS-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2795: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Introduce filesystem provisioner abstraction to Mesos containerizer > --- > > Key: MESOS-2795 > URL: https://issues.apache.org/jira/browse/MESOS-2795 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.22.1 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > Optional filesystem provisioner component for the Mesos containerizer that > can provision per-container filesystems. > This is different to a filesystem isolators because it just provisions a root > filesystem for a container and doesn't actually do any isolation (e.g., > through a mount namespace + pivot or chroot). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2794) Implement filesystem isolators
[ https://issues.apache.org/jira/browse/MESOS-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2794: -- Sprint: Twitter Q2 Sprint 3, Twitter Q2 Sprint 4 (was: Twitter Q2 Sprint 3) > Implement filesystem isolators > -- > > Key: MESOS-2794 > URL: https://issues.apache.org/jira/browse/MESOS-2794 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.22.1 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > Move persistent volume support from Mesos containerizer to separate > filesystem isolators, including support for container rootfs, where possible. > Use symlinks for posix systems without container rootfs. Use bind mounts for > Linux with/without container rootfs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2784) Add constexpr to C++11 whitelist
[ https://issues.apache.org/jira/browse/MESOS-2784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Brett updated MESOS-2784: -- Issue Type: Improvement (was: Bug) > Add constexpr to C++11 whitelist > > > Key: MESOS-2784 > URL: https://issues.apache.org/jira/browse/MESOS-2784 > Project: Mesos > Issue Type: Improvement > Components: documentation >Reporter: Paul Brett >Assignee: Paul Brett > Labels: twitter > > constexpr is currently used to eliminate initialization dependency issues for > non-POD objects. We should add it to the whitelist of acceptable c++11 > features in the style guide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
