[jira] [Commented] (MESOS-2340) Add ability to decode JSON serialized MasterInfo from ZK
[
https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653339#comment-14653339
]
Klaus Ma commented on MESOS-2340:
-
0.24 is next release, few months later.
> Add ability to decode JSON serialized MasterInfo from ZK
>
>
> Key: MESOS-2340
> URL: https://issues.apache.org/jira/browse/MESOS-2340
> Project: Mesos
> Issue Type: Improvement
> Components: leader election
>Reporter: Zameer Manji
>Assignee: Marco Massenzio
> Labels: mesosphere
> Fix For: 0.23.0
>
>
> Currently to discover the master a client needs the ZK node location and
> access to the MasterInfo protobuf so it can deserialize the binary blob in
> the node.
> I think it would be nice to publish JSON (like Twitter's ServerSets) so
> clients are not tied to protobuf to do service discovery.
> This ticket is an intermediate (compatibility) step: we add in {{0.23}} the
> ability for the {{Detector}} to "understand" JSON **alongside** Protobuf
> serialized format; this makes it compatible with both earlier versions, as
> well a future one (most likely, {{0.24}}) that will write the {{MasterInfo}}
> information in JSON format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2340) Add ability to decode JSON serialized MasterInfo from ZK
[
https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653343#comment-14653343
]
nicky paul sepnov commented on MESOS-2340:
--
thanks klaus,
i still trying install manually mesos from github repo with update using json.
> Add ability to decode JSON serialized MasterInfo from ZK
>
>
> Key: MESOS-2340
> URL: https://issues.apache.org/jira/browse/MESOS-2340
> Project: Mesos
> Issue Type: Improvement
> Components: leader election
>Reporter: Zameer Manji
>Assignee: Marco Massenzio
> Labels: mesosphere
> Fix For: 0.23.0
>
>
> Currently to discover the master a client needs the ZK node location and
> access to the MasterInfo protobuf so it can deserialize the binary blob in
> the node.
> I think it would be nice to publish JSON (like Twitter's ServerSets) so
> clients are not tied to protobuf to do service discovery.
> This ticket is an intermediate (compatibility) step: we add in {{0.23}} the
> ability for the {{Detector}} to "understand" JSON **alongside** Protobuf
> serialized format; this makes it compatible with both earlier versions, as
> well a future one (most likely, {{0.24}}) that will write the {{MasterInfo}}
> information in JSON format.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2073) Fetcher cache file verification, updating and invalidation
[ https://issues.apache.org/jira/browse/MESOS-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bernd Mathiske updated MESOS-2073: -- Shepherd: Benjamin Hindman > Fetcher cache file verification, updating and invalidation > -- > > Key: MESOS-2073 > URL: https://issues.apache.org/jira/browse/MESOS-2073 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave >Reporter: Bernd Mathiske >Assignee: Bernd Mathiske >Priority: Minor > Labels: mesosphere > Original Estimate: 96h > Remaining Estimate: 96h > > The other tickets in the fetcher cache epic do not necessitate a check sum > (e.g. MD5, SHA*) for files cached by the fetcher. Whereas such a check sum > could be used to verify whether the file arrived without unintended > alterations, it can first and foremost be employed to detect and trigger > updates. > Scenario: If a UIR is requested for fetching and the indicated download has > the same check sum as the cached file, then the cache file will be used and > the download forgone. If the check sum is different, then fetching proceeds > and the cached file gets replaced. > This capability will be indicated by an additional field in the URI protobuf. > Details TBD, i.e. to be discussed in comments below. > In addition to the above, even if the check sum is the same, we can support > voluntary cache file invalidation: a fresh download can be requested, or the > caching behavior can be revoked entirely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2073) Fetcher cache file verification, updating and invalidation
[ https://issues.apache.org/jira/browse/MESOS-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653531#comment-14653531 ] Bernd Mathiske commented on MESOS-2073: --- In phase 1, the server should enact dynamic content updates as follows. 1. Remove the checksum file. This will cause the fetcher to not cache content until we are in a stable situation again. 2. Swap in the new content file. 3. Put a fresh checksum file in place. This way there is no race between content and checksum preventing cache refreshing when it must be indicated. However, the saved checksum may not actually correspond to the downloaded content. It may be older. This is tolerable, because - in any case newer content than previously in the cache did get downloaded and - if the checksum is indeed out of sync, the next request will see either no checksum or a newer checksum and hence refresh the cache again. This means that in no case an update is missed. Worst case, the same content gets downloaded twice, once with the old checksum and then again with the correct one. This is also tolerable. In case multiple updates happen while holding on to an outdated checksum, this also works out OK, as long as every new checksum is different from all others. If the latter cannot be guaranteed, for instance because the checksum calculations in use are idempotent and old content may reappear, then we have a problem. Two approaches may help: A) Look at "Last-Modified" headers and such to still establish different identities of equal-looking checksums. B) Download the checksum before AND after downloading the content. Only if it matches, proceed. Otherwise start over. Approach A strongly hints that phase 1 would be obsolete if we simply honored HTTP cache directives. But this is a protocol-specific approach. I suggest that in the long run we implement both. But we start with phase 1, as this leads us to phase 2, which I reckon is what we typically really want. Note (again) that in phase 1 the checksum is allowed to say nothing about the integrity of the downloaded content. For example, simple serial numbers can be used. > Fetcher cache file verification, updating and invalidation > -- > > Key: MESOS-2073 > URL: https://issues.apache.org/jira/browse/MESOS-2073 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave >Reporter: Bernd Mathiske >Assignee: Bernd Mathiske >Priority: Minor > Labels: mesosphere > Original Estimate: 96h > Remaining Estimate: 96h > > The other tickets in the fetcher cache epic do not necessitate a check sum > (e.g. MD5, SHA*) for files cached by the fetcher. Whereas such a check sum > could be used to verify whether the file arrived without unintended > alterations, it can first and foremost be employed to detect and trigger > updates. > Scenario: If a UIR is requested for fetching and the indicated download has > the same check sum as the cached file, then the cache file will be used and > the download forgone. If the check sum is different, then fetching proceeds > and the cached file gets replaced. > This capability will be indicated by an additional field in the URI protobuf. > Details TBD, i.e. to be discussed in comments below. > In addition to the above, even if the check sum is the same, we can support > voluntary cache file invalidation: a fresh download can be requested, or the > caching behavior can be revoked entirely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3063) Add an example framework using dynamic reservation
[ https://issues.apache.org/jira/browse/MESOS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Ma reassigned MESOS-3063: --- Assignee: Klaus Ma > Add an example framework using dynamic reservation > -- > > Key: MESOS-3063 > URL: https://issues.apache.org/jira/browse/MESOS-3063 > Project: Mesos > Issue Type: Task >Reporter: Michael Park >Assignee: Klaus Ma > > An example framework using dynamic reservation should added to > # test dynamic reservations further, and > # to be used as a reference for those who want to use the dynamic reservation > feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3063) Add an example framework using dynamic reservation
[ https://issues.apache.org/jira/browse/MESOS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Klaus Ma updated MESOS-3063: Shepherd: Michael Park (was: Jie Yu) > Add an example framework using dynamic reservation > -- > > Key: MESOS-3063 > URL: https://issues.apache.org/jira/browse/MESOS-3063 > Project: Mesos > Issue Type: Task >Reporter: Michael Park >Assignee: Klaus Ma > > An example framework using dynamic reservation should added to > # test dynamic reservations further, and > # to be used as a reference for those who want to use the dynamic reservation > feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3063) Add an example framework using dynamic reservation
[ https://issues.apache.org/jira/browse/MESOS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653611#comment-14653611 ] Klaus Ma commented on MESOS-3063: - I'll work on this one, and try my best to finish it before 0.24 release. > Add an example framework using dynamic reservation > -- > > Key: MESOS-3063 > URL: https://issues.apache.org/jira/browse/MESOS-3063 > Project: Mesos > Issue Type: Task >Reporter: Michael Park >Assignee: Klaus Ma > > An example framework using dynamic reservation should added to > # test dynamic reservations further, and > # to be used as a reference for those who want to use the dynamic reservation > feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3073) Introduce HTTP endpoints for Quota
[
https://issues.apache.org/jira/browse/MESOS-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-3073:
---
Story Points: 3 (was: 5)
> Introduce HTTP endpoints for Quota
> --
>
> Key: MESOS-3073
> URL: https://issues.apache.org/jira/browse/MESOS-3073
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
> Labels: mesosphere
>
> We need to implement the HTTP endpoints for Quota as outlined in the Design
> Doc:
> (https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I).
> This also includes validating quota requests in terms of syntax correctness,
> updating Master bookkeeping structures, persisting quota requests in the
> {{Registry}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3199) Validate Quota Requests.
Joerg Schad created MESOS-3199: -- Summary: Validate Quota Requests. Key: MESOS-3199 URL: https://issues.apache.org/jira/browse/MESOS-3199 Project: Mesos Issue Type: Bug Reporter: Joerg Schad -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3073) Introduce HTTP endpoints for Quota
[
https://issues.apache.org/jira/browse/MESOS-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-3073:
---
Description:
We need to implement the HTTP endpoints for Quota as outlined in the Design
Doc:
(https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I).
was:
We need to implement the HTTP endpoints for Quota as outlined in the Design
Doc:
(https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I).
This also includes validating quota requests in terms of syntax correctness,
updating Master bookkeeping structures, persisting quota requests in the
{{Registry}}.
> Introduce HTTP endpoints for Quota
> --
>
> Key: MESOS-3073
> URL: https://issues.apache.org/jira/browse/MESOS-3073
> Project: Mesos
> Issue Type: Improvement
>Reporter: Joerg Schad
>Assignee: Joerg Schad
> Labels: mesosphere
>
> We need to implement the HTTP endpoints for Quota as outlined in the Design
> Doc:
> (https://docs.google.com/document/d/16iRNmziasEjVOblYp5bbkeBZ7pnjNlaIzPQqMTHQ-9I).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3199) Validate Quota Requests.
[
https://issues.apache.org/jira/browse/MESOS-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-3199:
---
Issue Type: Task (was: Bug)
> Validate Quota Requests.
>
>
> Key: MESOS-3199
> URL: https://issues.apache.org/jira/browse/MESOS-3199
> Project: Mesos
> Issue Type: Task
>Reporter: Joerg Schad
> Labels: mesosphere
>
> We need to validate quota requests in terms of syntax correctness, update
> Master bookkeeping structures, and persist quota requests in the {{Registry}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3199) Validate Quota Requests.
[
https://issues.apache.org/jira/browse/MESOS-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joerg Schad updated MESOS-3199:
---
Description: We need to validate quota requests in terms of syntax
correctness, update Master bookkeeping structures, and persist quota requests
in the {{Registry}}.
> Validate Quota Requests.
>
>
> Key: MESOS-3199
> URL: https://issues.apache.org/jira/browse/MESOS-3199
> Project: Mesos
> Issue Type: Bug
>Reporter: Joerg Schad
> Labels: mesosphere
>
> We need to validate quota requests in terms of syntax correctness, update
> Master bookkeeping structures, and persist quota requests in the {{Registry}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2073) Fetcher cache file verification, updating and invalidation
[ https://issues.apache.org/jira/browse/MESOS-2073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653655#comment-14653655 ] Bernd Mathiske commented on MESOS-2073: --- First patch in a series to implement phase 1: https://reviews.apache.org/r/37075/ > Fetcher cache file verification, updating and invalidation > -- > > Key: MESOS-2073 > URL: https://issues.apache.org/jira/browse/MESOS-2073 > Project: Mesos > Issue Type: Improvement > Components: fetcher, slave >Reporter: Bernd Mathiske >Assignee: Bernd Mathiske >Priority: Minor > Labels: mesosphere > Original Estimate: 96h > Remaining Estimate: 96h > > The other tickets in the fetcher cache epic do not necessitate a check sum > (e.g. MD5, SHA*) for files cached by the fetcher. Whereas such a check sum > could be used to verify whether the file arrived without unintended > alterations, it can first and foremost be employed to detect and trigger > updates. > Scenario: If a UIR is requested for fetching and the indicated download has > the same check sum as the cached file, then the cache file will be used and > the download forgone. If the check sum is different, then fetching proceeds > and the cached file gets replaced. > This capability will be indicated by an additional field in the URI protobuf. > Details TBD, i.e. to be discussed in comments below. > In addition to the above, even if the check sum is the same, we can support > voluntary cache file invalidation: a fresh download can be requested, or the > caching behavior can be revoked entirely. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3199) Validate Quota Requests.
[
https://issues.apache.org/jira/browse/MESOS-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Hindman updated MESOS-3199:
Sprint: Mesosphere Sprint 16
> Validate Quota Requests.
>
>
> Key: MESOS-3199
> URL: https://issues.apache.org/jira/browse/MESOS-3199
> Project: Mesos
> Issue Type: Task
>Reporter: Joerg Schad
> Labels: mesosphere
>
> We need to validate quota requests in terms of syntax correctness, update
> Master bookkeeping structures, and persist quota requests in the {{Registry}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-3199) Validate Quota Requests.
[
https://issues.apache.org/jira/browse/MESOS-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Hindman reassigned MESOS-3199:
---
Assignee: Joerg Schad
> Validate Quota Requests.
>
>
> Key: MESOS-3199
> URL: https://issues.apache.org/jira/browse/MESOS-3199
> Project: Mesos
> Issue Type: Task
>Reporter: Joerg Schad
>Assignee: Joerg Schad
> Labels: mesosphere
>
> We need to validate quota requests in terms of syntax correctness, update
> Master bookkeeping structures, and persist quota requests in the {{Registry}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2937) Initial design document for Quota support in Allocator.
[ https://issues.apache.org/jira/browse/MESOS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman updated MESOS-2937: Story Points: 5 (was: 8) > Initial design document for Quota support in Allocator. > --- > > Key: MESOS-2937 > URL: https://issues.apache.org/jira/browse/MESOS-2937 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Alexander Rukletsov >Assignee: Alexander Rukletsov > Labels: mesosphere > > Create a design document for the Quota feature support in the built-in > Hierarchical DRF allocator to be shared with the Mesos community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2937) Initial design document for Quota support in Allocator.
[ https://issues.apache.org/jira/browse/MESOS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman updated MESOS-2937: Summary: Initial design document for Quota support in Allocator. (was: Create a design document for Quota support in Allocator) > Initial design document for Quota support in Allocator. > --- > > Key: MESOS-2937 > URL: https://issues.apache.org/jira/browse/MESOS-2937 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Alexander Rukletsov >Assignee: Alexander Rukletsov > Labels: mesosphere > > Create a design document for the Quota feature support in the built-in > Hierarchical DRF allocator to be shared with the Mesos community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2937) Initial design document for Quota support in Allocator.
[ https://issues.apache.org/jira/browse/MESOS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman updated MESOS-2937: Sprint: Mesosphere Sprint 16 > Initial design document for Quota support in Allocator. > --- > > Key: MESOS-2937 > URL: https://issues.apache.org/jira/browse/MESOS-2937 > Project: Mesos > Issue Type: Documentation > Components: documentation >Reporter: Alexander Rukletsov >Assignee: Alexander Rukletsov > Labels: mesosphere > > Create a design document for the Quota feature support in the built-in > Hierarchical DRF allocator to be shared with the Mesos community. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2924) Allow simple construction via initializer list on hashset.
[
https://issues.apache.org/jira/browse/MESOS-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616383#comment-14616383
]
Benjamin Hindman edited comment on MESOS-2924 at 8/4/15 2:51 PM:
-
https://reviews.apache.org/r/35874/ <-- POC
was (Author: bernd-mesos):
https://reviews.apache.org/r/35874/
> Allow simple construction via initializer list on hashset.
> --
>
> Key: MESOS-2924
> URL: https://issues.apache.org/jira/browse/MESOS-2924
> Project: Mesos
> Issue Type: Improvement
>Reporter: Till Toenshoff
>Assignee: Alexander Rojas
>Priority: Minor
> Labels: mesosphere
>
> {{hashmap}} already has a initializer-list constructor, {{hashset}} should
> offer the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2924) Allow simple construction via initializer list on hashset.
[
https://issues.apache.org/jira/browse/MESOS-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Hindman updated MESOS-2924:
Shepherd: Bernd Mathiske
Sprint: Mesosphere Sprint 16
> Allow simple construction via initializer list on hashset.
> --
>
> Key: MESOS-2924
> URL: https://issues.apache.org/jira/browse/MESOS-2924
> Project: Mesos
> Issue Type: Improvement
>Reporter: Till Toenshoff
>Assignee: Alexander Rojas
>Priority: Minor
> Labels: mesosphere
>
> {{hashmap}} already has a initializer-list constructor, {{hashset}} should
> offer the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653804#comment-14653804
]
haosdent commented on MESOS-2337:
-
Hi, [~marco-mesos] "__init__.py not getting installed in
$PREFIX/lib/pythonX.Y/site-packages/mesos" is correct, because we use multi
submodule way to package python. I think this ticket is not a problem.
Under src/python directory, we have native/protocol/interface modules. They
share a same parent mesos. If we contains __init__.py under mesos, it would
conflict when install. Because every python submodule have a __init__.py under
mesos. So python setup tool would skip this when package them.
{quote}
Skipping installation of build/bdist.linux-x86_64/wheel/mesos/__init__.py
(namespace package)
{quote}
And python would use something like mesos.interface-0.23.0-py2.6-nspkg.pth to
make sure mesos could be import correctly, although we don't have __init__.py
under $PREFIX/lib/pythonX.Y/site-packages/mesos.
But pth seems only could load from default sys.path or add it to
site.addsitedir. If you want to load it through set $PYTHONPATH, it could not
work.
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653813#comment-14653813
]
James Peach commented on MESOS-2337:
Seems related to MESOS-3198 or maybe a duplicate. AFAICT none of the
python-based tools work at all in a default installation.
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653851#comment-14653851
]
haosdent commented on MESOS-2337:
-
Or we could change src/python/src/meso/__init__.py and let it as the only
__init__.py file which install to $PREFIX/lib/pythonX.Y/site-packages/mesos?
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653859#comment-14653859
]
haosdent commented on MESOS-2337:
-
Refrence openstack, them have
/usr/lib/python2.7/dist-packages/openstack/__init__.py under [common
module|http://packages.ubuntu.com/zh-cn/precise/amd64/python-openstack-common/filelist],
and other [sub
module|http://packages.ubuntu.com/zh-cn/precise/amd64/python-openstack-compute/filelist]
don't contains this.
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3196) Always set TaskStatus.executor_id when sending a status update message from Executor
[ https://issues.apache.org/jira/browse/MESOS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653899#comment-14653899 ] Kapil Arya commented on MESOS-3196: --- The idea was to fill it in the executor process as we are already filling in some other information. However, we are doing the same in the Slave as well, so we can very well have the Slave set it. > Always set TaskStatus.executor_id when sending a status update message from > Executor > > > Key: MESOS-3196 > URL: https://issues.apache.org/jira/browse/MESOS-3196 > Project: Mesos > Issue Type: Task >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > > Currently, the Executor doesn't always set TaskStatus.executor_id. This > prevents the Slave TaskStatus label decorator hook from knowing the executor > id. > An appropriate place to automatically fill in the executor_id is > ExecutorProcesS::sendStatusUpdate() since we are already filling in some > other information here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653939#comment-14653939
]
haosdent commented on MESOS-2337:
-
A draft patch for this: https://reviews.apache.org/r/37081/diff/1#index_header
. I only test in on CentOS 6.6, need test in other OS.
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2337) __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
[
https://issues.apache.org/jira/browse/MESOS-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14653939#comment-14653939
]
haosdent edited comment on MESOS-2337 at 8/4/15 5:22 PM:
-
A draft patch for this: https://reviews.apache.org/r/37081 . I only test in on
CentOS 6.6, need test in other OS.
was (Author: haosd...@gmail.com):
A draft patch for this: https://reviews.apache.org/r/37081/diff/1#index_header
. I only test in on CentOS 6.6, need test in other OS.
> __init__.py not getting installed in $PREFIX/lib/pythonX.Y/site-packages/mesos
> --
>
> Key: MESOS-2337
> URL: https://issues.apache.org/jira/browse/MESOS-2337
> Project: Mesos
> Issue Type: Bug
> Components: build, python api
>Reporter: Kapil Arya
>Assignee: Marco Massenzio
>Priority: Critical
> Labels: mesosphere
>
> When doing a {{make install}}, the src/python/native/src/mesos/__init__.py
> file is not getting installed in
> {{$PREFIX/lib/pythonX.Y/site-packages/mesos/}}.
> This makes it impossible to do the following import when {{PYTHONPATH}} is
> set to the {{site-packages}} directory.
> {code}
> import mesos.interface.mesos_pb2
> {code}
> The directories {{$PREFIX/lib/pythonX.Y/site-packages/mesos/interface,
> native}} do have their corresponding {{__init__.py}} files.
> Reproducing the bug:
> {code}
> ../configure --prefix=$HOME/test-install && make install
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3196) Always set TaskStatus.executor_id when sending a status update message from Executor
[ https://issues.apache.org/jira/browse/MESOS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654010#comment-14654010 ] Vinod Kone commented on MESOS-3196: --- As we move towards HTTP API for slave <-> executor communication, it is imperative that any smarts be implemented in the slave (whenever possible) instead of the executor driver (which will be deprecated). > Always set TaskStatus.executor_id when sending a status update message from > Executor > > > Key: MESOS-3196 > URL: https://issues.apache.org/jira/browse/MESOS-3196 > Project: Mesos > Issue Type: Task >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > > Currently, the Executor doesn't always set TaskStatus.executor_id. This > prevents the Slave TaskStatus label decorator hook from knowing the executor > id. > An appropriate place to automatically fill in the executor_id is > ExecutorProcesS::sendStatusUpdate() since we are already filling in some > other information here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2924) Allow simple construction via initializer list on hashset.
[
https://issues.apache.org/jira/browse/MESOS-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Park updated MESOS-2924:
Shepherd: Michael Park (was: Bernd Mathiske)
> Allow simple construction via initializer list on hashset.
> --
>
> Key: MESOS-2924
> URL: https://issues.apache.org/jira/browse/MESOS-2924
> Project: Mesos
> Issue Type: Improvement
>Reporter: Till Toenshoff
>Assignee: Alexander Rojas
>Priority: Minor
> Labels: mesosphere
>
> {{hashmap}} already has a initializer-list constructor, {{hashset}} should
> offer the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3166) Design doc for docker image registry
[ https://issues.apache.org/jira/browse/MESOS-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jojy Varghese updated MESOS-3166: - Summary: Design doc for docker image registry (was: Design doc for docker image registry authenticator) > Design doc for docker image registry > > > Key: MESOS-3166 > URL: https://issues.apache.org/jira/browse/MESOS-3166 > Project: Mesos > Issue Type: Bug > Components: containerization > Environment: linux >Reporter: Jojy Varghese >Assignee: Jojy Varghese > Labels: mesosphere > > Create design document for the docker registry Authenticator component so > that we have a baseline for the implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3166) Design doc for docker image registry authenticator
[ https://issues.apache.org/jira/browse/MESOS-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jojy Varghese updated MESOS-3166: - Summary: Design doc for docker image registry authenticator (was: Design doc for docker image registry) > Design doc for docker image registry authenticator > -- > > Key: MESOS-3166 > URL: https://issues.apache.org/jira/browse/MESOS-3166 > Project: Mesos > Issue Type: Bug > Components: containerization > Environment: linux >Reporter: Jojy Varghese >Assignee: Jojy Varghese > Labels: mesosphere > > Create design document for the docker registry Authenticator component so > that we have a baseline for the implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2294) Implement the Events stream on master for Call endpoint
[ https://issues.apache.org/jira/browse/MESOS-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654020#comment-14654020 ] Anand Mazumdar commented on MESOS-2294: --- Tests for subscribe/failover workflow: https://reviews.apache.org/r/37082 > Implement the Events stream on master for Call endpoint > --- > > Key: MESOS-2294 > URL: https://issues.apache.org/jira/browse/MESOS-2294 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Anand Mazumdar > Labels: mesosphere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3198) mesos.native could not found
[ https://issues.apache.org/jira/browse/MESOS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654023#comment-14654023 ] haosdent commented on MESOS-3198: - hi, [~jamespeach] /usr/libexec/mesos/python is related to this issue [MESOS-3149|https://issues.apache.org/jira/browse/MESOS-3149] > mesos.native could not found > > > Key: MESOS-3198 > URL: https://issues.apache.org/jira/browse/MESOS-3198 > Project: Mesos > Issue Type: Bug > Components: build >Affects Versions: 0.23.0 > Environment: Ubuntu14.04 Mesos0.23.0 >Reporter: pugna > Labels: build, newbie, python > Attachments: Re mesos.native could not found.msg > > > I deploy apache mesos-0.23 on Ubuntu14.04 This error comes from the last step > "# Run Python framework (Exits after successfully running some tasks.). $ > ./src/examples/python/test-framework 127.0.0.1:5050" > Mesos/src/examples/python/test_framework.py line 25, mesos.native could not > found > Anyone who can help me solve this problem? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3200) Remove unused 'fatal' and 'fatalerror' macros
Michael Park created MESOS-3200:
---
Summary: Remove unused 'fatal' and 'fatalerror' macros
Key: MESOS-3200
URL: https://issues.apache.org/jira/browse/MESOS-3200
Project: Mesos
Issue Type: Task
Components: libprocess, stout
Affects Versions: 0.23.0
Reporter: Michael Park
There exist {{fatal}} and {{fatalerror}} macros in both {{libprocess}} and
{{stout}}. None of them are currently used as we favor {{glog}}'s
{{LOG(FATAL)}}, and therefore should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-3200) Remove unused 'fatal' and 'fatalerror' macros
[
https://issues.apache.org/jira/browse/MESOS-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Park reassigned MESOS-3200:
---
Assignee: Michael Park
> Remove unused 'fatal' and 'fatalerror' macros
> -
>
> Key: MESOS-3200
> URL: https://issues.apache.org/jira/browse/MESOS-3200
> Project: Mesos
> Issue Type: Task
> Components: libprocess, stout
>Affects Versions: 0.23.0
>Reporter: Michael Park
>Assignee: Michael Park
>
> There exist {{fatal}} and {{fatalerror}} macros in both {{libprocess}} and
> {{stout}}. None of them are currently used as we favor {{glog}}'s
> {{LOG(FATAL)}}, and therefore should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3200) Remove unused 'fatal' and 'fatalerror' macros
[
https://issues.apache.org/jira/browse/MESOS-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654028#comment-14654028
]
Michael Park commented on MESOS-3200:
-
stout: https://reviews.apache.org/r/37010/
libprocess: https://reviews.apache.org/r/37011/
mesos: https://reviews.apache.org/r/37012/
> Remove unused 'fatal' and 'fatalerror' macros
> -
>
> Key: MESOS-3200
> URL: https://issues.apache.org/jira/browse/MESOS-3200
> Project: Mesos
> Issue Type: Task
> Components: libprocess, stout
>Affects Versions: 0.23.0
>Reporter: Michael Park
>
> There exist {{fatal}} and {{fatalerror}} macros in both {{libprocess}} and
> {{stout}}. None of them are currently used as we favor {{glog}}'s
> {{LOG(FATAL)}}, and therefore should be removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3196) Always set TaskStatus.executor_id when sending a status update message from Executor
[ https://issues.apache.org/jira/browse/MESOS-3196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654043#comment-14654043 ] Niklas Quarfot Nielsen commented on MESOS-3196: --- I am fine with either; [~karya] how about doing it in Slave::statusUpdate? We ensure that the source is set to 'slave' and could set the executor_id accordingly > Always set TaskStatus.executor_id when sending a status update message from > Executor > > > Key: MESOS-3196 > URL: https://issues.apache.org/jira/browse/MESOS-3196 > Project: Mesos > Issue Type: Task >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > > Currently, the Executor doesn't always set TaskStatus.executor_id. This > prevents the Slave TaskStatus label decorator hook from knowing the executor > id. > An appropriate place to automatically fill in the executor_id is > ExecutorProcesS::sendStatusUpdate() since we are already filling in some > other information here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3201) Libev handle_async can deadlock with run_in_event_loop
Joris Van Remoortere created MESOS-3201:
---
Summary: Libev handle_async can deadlock with run_in_event_loop
Key: MESOS-3201
URL: https://issues.apache.org/jira/browse/MESOS-3201
Project: Mesos
Issue Type: Bug
Components: libprocess
Affects Versions: 0.23.0
Reporter: Joris Van Remoortere
Assignee: Joris Van Remoortere
Due to the arbitrary nature of the functions that are executed in handle_async,
invoking them under the (A) {{watchers_mutex}} can lead to deadlocks if (B) is
acquired before calling {{run_in_event_loop}} and (B) is also acquired within
the arbitrary function.
{code}
==82679== Thread #10: lock order "0x60774F8 before 0x60768C0" violated
==82679==
==82679== Observed (incorrect) order is: acquisition of lock at 0x60768C0
==82679==at 0x4C32145: pthread_mutex_lock (in
/usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==82679==by 0x692C9B: __gthread_mutex_lock(pthread_mutex_t*)
(gthr-default.h:748)
==82679==by 0x6950BF: std::mutex::lock() (mutex:134)
==82679==by 0x696219: Synchronized
synchronize(std::mutex*)::{lambda(std::mutex*)#1}::operator()(std::mutex*)
const (synchronized.hpp:58)
==82679==by 0x696238: Synchronized
synchronize(std::mutex*)::{lambda(std::mutex*)#1}::_FUN(std::mutex*)
(synchronized.hpp:58)
==82679==by 0x6984CF: Synchronized::Synchronized(std::mutex*,
void (*)(std::mutex*), void (*)(std::mutex*)) (synchronized.hpp:35)
==82679==by 0x6962DE: Synchronized
synchronize(std::mutex*) (synchronized.hpp:60)
==82679==by 0x728FE1: process::handle_async(ev_loop*, ev_async*, int)
(libev.cpp:48)
==82679==by 0x761384: ev_invoke_pending (ev.c:2994)
==82679==by 0x7643C4: ev_run (ev.c:3394)
==82679==by 0x728E37: ev_loop (ev.h:826)
==82679==by 0x729469: process::EventLoop::run() (libev.cpp:135)
==82679==
==82679== followed by a later acquisition of lock at 0x60774F8
==82679==at 0x4C32145: pthread_mutex_lock (in
/usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==82679==by 0x4C6F9D: __gthread_mutex_lock(pthread_mutex_t*)
(gthr-default.h:748)
==82679==by 0x4C6FED: __gthread_recursive_mutex_lock(pthread_mutex_t*)
(gthr-default.h:810)
==82679==by 0x4F5D3D: std::recursive_mutex::lock() (mutex:175)
==82679==by 0x516513: Synchronized
synchronize(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::operator()(std::recursive_mutex*)
const (synchronized.hpp:58)
==82679==by 0x516532: Synchronized
synchronize(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::_FUN(std::recursive_mutex*)
(synchronized.hpp:58)
==82679==by 0x52E619:
Synchronized::Synchronized(std::recursive_mutex*, void
(*)(std::recursive_mutex*), void (*)(std::recursive_mutex*))
(synchronized.hpp:35)
==82679==by 0x5165D4: Synchronized
synchronize(std::recursive_mutex*) (synchronized.hpp:60)
==82679==by 0x6BF4E1: process::ProcessManager::use(process::UPID const&)
(process.cpp:2127)
==82679==by 0x6C2B8C: process::ProcessManager::terminate(process::UPID
const&, bool, process::ProcessBase*) (process.cpp:2604)
==82679==by 0x6C6C3C: process::terminate(process::UPID const&, bool)
(process.cpp:3107)
==82679==by 0x692B65: process::Latch::trigger() (latch.cpp:53)
{code}
This was introduced in
https://github.com/apache/mesos/commit/849fc4d361e40062073324153ba97e98e294fdf2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3187) Docker cli option support
[ https://issues.apache.org/jira/browse/MESOS-3187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654070#comment-14654070 ] Vaibhav Khanduja commented on MESOS-3187: - The mesos slave cli option will be enhanced to support docker daemon host parameter. The new cli option will be "docker_host", taking in parameter similar to docker daemon. For e.g. if docker daemon has been started with non-default unix socket unix::///var/run/docker_mydefault.sock, the slave can be started with a similar option --docker_host=unix::///var/run/docker_mydefault.sock .. The slave boot up checks for version of the docker, this will validate the correctness of the parameter. > Docker cli option support > - > > Key: MESOS-3187 > URL: https://issues.apache.org/jira/browse/MESOS-3187 > Project: Mesos > Issue Type: Improvement > Components: docker, slave >Reporter: Vaibhav Khanduja >Assignee: Vaibhav Khanduja >Priority: Minor > > Mesos slave today support docker as a container environment. The docker cli > support much more options than what is supported by mesos slave. The slave > command line option should be enhanced support such parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3202) Avoid frameworks starving in DRF allocator.
Joerg Schad created MESOS-3202: -- Summary: Avoid frameworks starving in DRF allocator. Key: MESOS-3202 URL: https://issues.apache.org/jira/browse/MESOS-3202 Project: Mesos Issue Type: Bug Reporter: Joerg Schad We currently run into issues with the DRF scheduler that frameworks do not receive offers (see https://github.com/mesosphere/marathon/issues/1931 for details). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3139) Incorporate CMake into standard documentation
[ https://issues.apache.org/jira/browse/MESOS-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654120#comment-14654120 ] haosdent commented on MESOS-3139: - For glog/libev/protobuf/http-parser, I build it failed through visual studio. Do we need custom CMakeFile to these projects? > Incorporate CMake into standard documentation > - > > Key: MESOS-3139 > URL: https://issues.apache.org/jira/browse/MESOS-3139 > Project: Mesos > Issue Type: Task > Components: build >Reporter: Alex Clemmer >Assignee: Alex Clemmer > Labels: build, cmake, mesosphere > > Right now it's anyone's guess how to build with CMake. If we want people to > use it, we should put up documentation. The central challenge is that the > CMake instructions will be slightly different for different platforms. > For example, on Linux, the gist of the build is basically the same as > autotools; you pull down the system dependencies (like APR, _etc_.), and then: > ``` > ./bootstrap > mkdir build-cmake && cd build-cmake > cmake .. > make > ``` > But, on Windows, it will be somewhat more complicated. There is no bootstrap > step, for example, because Windows doesn't have bash natively. And even when > we put that in, you'll still have to build the glog stuff out-of-band because > CMake has no way of booting up Visual Studio and calling "build." > So practically, we need to figure out: > * What our build story is for different platforms > * Write specific instructions for our "core" target platforms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3202) Avoid frameworks starving in DRF allocator.
[ https://issues.apache.org/jira/browse/MESOS-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654116#comment-14654116 ] Vinod Kone commented on MESOS-3202: --- Can you expand on the description? AFAICT, that github link doesn't provide any details about what the problem is. > Avoid frameworks starving in DRF allocator. > --- > > Key: MESOS-3202 > URL: https://issues.apache.org/jira/browse/MESOS-3202 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad > > We currently run into issues with the DRF scheduler that frameworks do not > receive offers (see https://github.com/mesosphere/marathon/issues/1931 for > details). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3202) Avoid frameworks starving in DRF allocator.
[ https://issues.apache.org/jira/browse/MESOS-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joerg Schad updated MESOS-3202: --- Description: We currently run into issues with the DRF scheduler that frameworks do not receive offers (see https://github.com/mesosphere/marathon/issues/1931 for details). Imagine that we have 10 frameworks and unallocated resources from a single slave. Allocation interval is 1 sec, and refuse_seconds (i.e. the time for which a declined resource is filtered) is 3 sec across all frameworks. Allocator offers resources to framework 1 (according to DRF) which declines the offer immediately. In the next allocation interval framework 1 is skipped due to the declined offer before. Hence the next framework 2 is offered the resources, which it also declines. The same procedure in the next allocation interval (with framework 3). In the next allocation interval the refuse_seconds for framework 1 are over, and as it still has the lowest DRF share it gets the resource offered again, which it again declines. And the cycle begins again Framework 4 (which is actually waiting for this resource) is never offered this resource. was:We currently run into issues with the DRF scheduler that frameworks do not receive offers (see https://github.com/mesosphere/marathon/issues/1931 for details). > Avoid frameworks starving in DRF allocator. > --- > > Key: MESOS-3202 > URL: https://issues.apache.org/jira/browse/MESOS-3202 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad > > We currently run into issues with the DRF scheduler that frameworks do not > receive offers (see https://github.com/mesosphere/marathon/issues/1931 for > details). > Imagine that we have 10 frameworks and unallocated resources from a single > slave. > Allocation interval is 1 sec, and refuse_seconds (i.e. the time for which a > declined resource is filtered) is 3 sec across all frameworks. > Allocator offers resources to framework 1 (according to DRF) which declines > the offer immediately. > In the next allocation interval framework 1 is skipped due to the declined > offer before. Hence the next framework 2 is offered the resources, which it > also declines. > The same procedure in the next allocation interval (with framework 3). > In the next allocation interval the refuse_seconds for framework 1 are over, > and as it still has the lowest DRF share it gets the resource offered again, > which it again declines. And the cycle begins again > Framework 4 (which is actually waiting for this resource) is never offered > this resource. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3139) Incorporate CMake into standard documentation
[ https://issues.apache.org/jira/browse/MESOS-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654305#comment-14654305 ] Alex Clemmer commented on MESOS-3139: - I'm not 100% sure what you mean, but I think you're saying that the third-party libraries don't build on Windows. That's true; I have figured out the quirks of glog and protobuf, but haven't written them down yet. If you look at my recent reviews(e.g., [1]), you will see that we're just now getting to a place where glog will be convenient to build on Windows. [1] https://reviews.apache.org/r/37020/ > Incorporate CMake into standard documentation > - > > Key: MESOS-3139 > URL: https://issues.apache.org/jira/browse/MESOS-3139 > Project: Mesos > Issue Type: Task > Components: build >Reporter: Alex Clemmer >Assignee: Alex Clemmer > Labels: build, cmake, mesosphere > > Right now it's anyone's guess how to build with CMake. If we want people to > use it, we should put up documentation. The central challenge is that the > CMake instructions will be slightly different for different platforms. > For example, on Linux, the gist of the build is basically the same as > autotools; you pull down the system dependencies (like APR, _etc_.), and then: > ``` > ./bootstrap > mkdir build-cmake && cd build-cmake > cmake .. > make > ``` > But, on Windows, it will be somewhat more complicated. There is no bootstrap > step, for example, because Windows doesn't have bash natively. And even when > we put that in, you'll still have to build the glog stuff out-of-band because > CMake has no way of booting up Visual Studio and calling "build." > So practically, we need to figure out: > * What our build story is for different platforms > * Write specific instructions for our "core" target platforms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3192) ContainerInfo::Image::AppC::id should be optional
[
https://issues.apache.org/jira/browse/MESOS-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Xu updated MESOS-3192:
--
Sprint: Twitter Mesos Q3 Sprint 3
Story Points: 1
> ContainerInfo::Image::AppC::id should be optional
> -
>
> Key: MESOS-3192
> URL: https://issues.apache.org/jira/browse/MESOS-3192
> Project: Mesos
> Issue Type: Bug
>Reporter: Yan Xu
>
> As I commented here: https://reviews.apache.org/r/34136/
> Currently ContainerInfo::Image::Appc is defined as the following
> {noformat:title=}
> message AppC {
> required string name = 1;
> required string id = 2;
> optional Labels labels = 3;
> }
> {noformat}
> In which the {{id}} is a required field. When users specify the image in
> tasks they likely will not use an image id and we should change it to be
> optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3191) Implement a utility for computing hash in libprocess
[ https://issues.apache.org/jira/browse/MESOS-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-3191: -- Sprint: Twitter Mesos Q3 Sprint 3 Story Points: 2 > Implement a utility for computing hash in libprocess > > > Key: MESOS-3191 > URL: https://issues.apache.org/jira/browse/MESOS-3191 > Project: Mesos > Issue Type: Task >Reporter: Yan Xu > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3193) Implement AppC image discovery.
[ https://issues.apache.org/jira/browse/MESOS-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu reassigned MESOS-3193: - Assignee: Yan Xu > Implement AppC image discovery. > --- > > Key: MESOS-3193 > URL: https://issues.apache.org/jira/browse/MESOS-3193 > Project: Mesos > Issue Type: Task >Reporter: Yan Xu >Assignee: Yan Xu > Labels: twitter > > https://reviews.apache.org/r/34139/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3192) ContainerInfo::Image::AppC::id should be optional
[
https://issues.apache.org/jira/browse/MESOS-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Xu reassigned MESOS-3192:
-
Assignee: Yan Xu
> ContainerInfo::Image::AppC::id should be optional
> -
>
> Key: MESOS-3192
> URL: https://issues.apache.org/jira/browse/MESOS-3192
> Project: Mesos
> Issue Type: Bug
>Reporter: Yan Xu
>Assignee: Yan Xu
>
> As I commented here: https://reviews.apache.org/r/34136/
> Currently ContainerInfo::Image::Appc is defined as the following
> {noformat:title=}
> message AppC {
> required string name = 1;
> required string id = 2;
> optional Labels labels = 3;
> }
> {noformat}
> In which the {{id}} is a required field. When users specify the image in
> tasks they likely will not use an image id and we should change it to be
> optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-3191) Implement a utility for computing hash in libprocess
[ https://issues.apache.org/jira/browse/MESOS-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu reassigned MESOS-3191: - Assignee: Yan Xu > Implement a utility for computing hash in libprocess > > > Key: MESOS-3191 > URL: https://issues.apache.org/jira/browse/MESOS-3191 > Project: Mesos > Issue Type: Task >Reporter: Yan Xu >Assignee: Yan Xu > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3193) Implement AppC image discovery.
[ https://issues.apache.org/jira/browse/MESOS-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-3193: -- Sprint: Twitter Mesos Q3 Sprint 3 Story Points: 2 Labels: twitter (was: ) > Implement AppC image discovery. > --- > > Key: MESOS-3193 > URL: https://issues.apache.org/jira/browse/MESOS-3193 > Project: Mesos > Issue Type: Task >Reporter: Yan Xu > Labels: twitter > > https://reviews.apache.org/r/34139/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3191) Implement a utility for computing hash in libprocess
[ https://issues.apache.org/jira/browse/MESOS-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-3191: -- Labels: twitter (was: ) > Implement a utility for computing hash in libprocess > > > Key: MESOS-3191 > URL: https://issues.apache.org/jira/browse/MESOS-3191 > Project: Mesos > Issue Type: Task >Reporter: Yan Xu >Assignee: Yan Xu > Labels: twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3192) ContainerInfo::Image::AppC::id should be optional
[
https://issues.apache.org/jira/browse/MESOS-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yan Xu updated MESOS-3192:
--
Labels: twitter (was: )
> ContainerInfo::Image::AppC::id should be optional
> -
>
> Key: MESOS-3192
> URL: https://issues.apache.org/jira/browse/MESOS-3192
> Project: Mesos
> Issue Type: Bug
>Reporter: Yan Xu
>Assignee: Yan Xu
> Labels: twitter
>
> As I commented here: https://reviews.apache.org/r/34136/
> Currently ContainerInfo::Image::Appc is defined as the following
> {noformat:title=}
> message AppC {
> required string name = 1;
> required string id = 2;
> optional Labels labels = 3;
> }
> {noformat}
> In which the {{id}} is a required field. When users specify the image in
> tasks they likely will not use an image id and we should change it to be
> optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2916) Expose State API via HTTP
[ https://issues.apache.org/jira/browse/MESOS-2916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654366#comment-14654366 ] Adam B commented on MESOS-2916: --- Does this refer to the "state abstraction" for zk/replicated log, or does it refer to `state.json`? > Expose State API via HTTP > - > > Key: MESOS-2916 > URL: https://issues.apache.org/jira/browse/MESOS-2916 > Project: Mesos > Issue Type: Story >Reporter: Tomás Senart > Labels: http > > The State API is a useful service for frameworks to use. It would make sense > to have it available via the public HTTP API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2847) User can specify image provisioners and provisioning backends
[
https://issues.apache.org/jira/browse/MESOS-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654458#comment-14654458
]
Yan Xu edited comment on MESOS-2847 at 8/4/15 10:20 PM:
[~tnachen] There is already {{\-\-provisioners}} and for AppC the code in
review has {{\-\-appc_backend}}. In a unified scenario the backend can be
specified via {{\-\-provisioner_backend}} and we need to figure out
compatibility between the two flags. It looks like in most cases whatever
backend AppC can use should also be Docker compatible.
Is this what the ticket is intended to track?
was (Author: xujyan):
[~tnachen] There is already {{--provisioners}} and for AppC the code in review
has {{--appc_backend}}. In a unified scenario the backend can be specified via
{{--provisioner_backend}} and we need to figure out compatibility between the
two flags. It looks like in most cases whatever backend AppC can use should
also be Docker compatible.
Is this what the ticket is intended to track?
> User can specify image provisioners and provisioning backends
> -
>
> Key: MESOS-2847
> URL: https://issues.apache.org/jira/browse/MESOS-2847
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: Timothy Chen
>Assignee: Ian Downes
> Labels: unified-prototype
>
> Support new flags so users can pass in provisioners and backends
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2847) User can specify image provisioners and provisioning backends
[
https://issues.apache.org/jira/browse/MESOS-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654458#comment-14654458
]
Yan Xu commented on MESOS-2847:
---
[~tnachen] There is already {{--provisioners}} and for AppC the code in review
has {{--appc_backend}}. In a unified scenario the backend can be specified via
{{--provisioner_backend}} and we need to figure out compatibility between the
two flags. It looks like in most cases whatever backend AppC can use should
also be Docker compatible.
Is this what the ticket is intended to track?
> User can specify image provisioners and provisioning backends
> -
>
> Key: MESOS-2847
> URL: https://issues.apache.org/jira/browse/MESOS-2847
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: Timothy Chen
>Assignee: Ian Downes
> Labels: unified-prototype
>
> Support new flags so users can pass in provisioners and backends
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3203) MasterAuthorizationTest.DuplicateRegistration test is flaky
Vinod Kone created MESOS-3203: - Summary: MasterAuthorizationTest.DuplicateRegistration test is flaky Key: MESOS-3203 URL: https://issues.apache.org/jira/browse/MESOS-3203 Project: Mesos Issue Type: Bug Components: test Environment: https://builds.apache.org/job/Mesos/COMPILER=gcc,CONFIGURATION=--verbose,OS=ubuntu:14.04,label_exp=docker%7C%7CHadoop/639/consoleFull Reporter: Vinod Kone Assignee: Vinod Kone [ RUN ] MasterAuthorizationTest.DuplicateRegistration Using temporary directory '/tmp/MasterAuthorizationTest_DuplicateRegistration_NKT3f7' I0804 22:16:01.578500 26185 leveldb.cpp:176] Opened db in 2.188338ms I0804 22:16:01.579172 26185 leveldb.cpp:183] Compacted db in 645075ns I0804 22:16:01.579211 26185 leveldb.cpp:198] Created db iterator in 15766ns I0804 22:16:01.579227 26185 leveldb.cpp:204] Seeked to beginning of db in 1658ns I0804 22:16:01.579238 26185 leveldb.cpp:273] Iterated through 0 keys in the db in 313ns I0804 22:16:01.579282 26185 replica.cpp:744] Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned I0804 22:16:01.579787 26212 recover.cpp:449] Starting replica recovery I0804 22:16:01.580075 26212 recover.cpp:475] Replica is in EMPTY status I0804 22:16:01.581014 26205 replica.cpp:641] Replica in EMPTY status received a broadcasted recover request I0804 22:16:01.581357 26211 recover.cpp:195] Received a recover response from a replica in EMPTY status I0804 22:16:01.581761 26207 recover.cpp:566] Updating replica status to STARTING I0804 22:16:01.582334 26218 master.cpp:377] Master 20150804-221601-2550141356-59302-26185 (d6d349cd895b) started on 172.17.0.152:59302 I0804 22:16:01.582355 26218 master.cpp:379] Flags at startup: --acls="" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate="true" --authenticate_slaves="true" --authenticators="crammd5" --credentials="/tmp/MasterAuthorizationTest_DuplicateRegistration_NKT3f7/credentials" --framework_sorter="drf" --help="false" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5" --quiet="false" --recovery_slave_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="25secs" --registry_strict="true" --root_submissions="true" --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins" --user_sorter="drf" --version="false" --webui_dir="/mesos/mesos-0.24.0/_inst/share/mesos/webui" --work_dir="/tmp/MasterAuthorizationTest_DuplicateRegistration_NKT3f7/master" --zk_session_timeout="10secs" I0804 22:16:01.582711 26218 master.cpp:424] Master only allowing authenticated frameworks to register I0804 22:16:01.582722 26218 master.cpp:429] Master only allowing authenticated slaves to register I0804 22:16:01.582728 26218 credentials.hpp:37] Loading credentials for authentication from '/tmp/MasterAuthorizationTest_DuplicateRegistration_NKT3f7/credentials' I0804 22:16:01.582929 26204 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 421543ns I0804 22:16:01.582950 26204 replica.cpp:323] Persisted replica status to STARTING I0804 22:16:01.583032 26218 master.cpp:468] Using default 'crammd5' authenticator I0804 22:16:01.583132 26211 recover.cpp:475] Replica is in STARTING status I0804 22:16:01.583154 26218 master.cpp:505] Authorization enabled I0804 22:16:01.583356 26214 whitelist_watcher.cpp:79] No whitelist given I0804 22:16:01.583411 26217 hierarchical.hpp:346] Initialized hierarchical allocator process I0804 22:16:01.583976 26213 replica.cpp:641] Replica in STARTING status received a broadcasted recover request I0804 22:16:01.584187 26209 recover.cpp:195] Received a recover response from a replica in STARTING status I0804 22:16:01.584581 26213 master.cpp:1495] The newly elected leader is master@172.17.0.152:59302 with id 20150804-221601-2550141356-59302-26185 I0804 22:16:01.584609 26213 master.cpp:1508] Elected as the leading master! I0804 22:16:01.584627 26213 master.cpp:1278] Recovering from registrar I0804 22:16:01.584656 26204 recover.cpp:566] Updating replica status to VOTING I0804 22:16:01.584770 26212 registrar.cpp:313] Recovering registrar I0804 22:16:01.585261 26218 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 370526ns I0804 22:16:01.585285 26218 replica.cpp:323] Persisted replica status to VOTING I0804 22:16:01.585412 26216 recover.cpp:580] Successfully joined the Paxos group I0804 22:16:01.585667 26216 recover.cpp:464] Recover process terminated I0804 2
[jira] [Created] (MESOS-3204) PortMappingIsolatorProcess shell script can silently fail
Paul Brett created MESOS-3204: - Summary: PortMappingIsolatorProcess shell script can silently fail Key: MESOS-3204 URL: https://issues.apache.org/jira/browse/MESOS-3204 Project: Mesos Issue Type: Bug Components: isolation Affects Versions: 0.24.0 Reporter: Paul Brett Assignee: Paul Brett PortMappingIsolatorProcess::scripts generates a shell script to configure the target environment but does not set the shell '-e' flag. Hence errors generated by the script will be silently ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-830) ExamplesTest.JavaFramework is flaky
[
https://issues.apache.org/jira/browse/MESOS-830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654519#comment-14654519
]
Greg Mann commented on MESOS-830:
-
It seems this issue is due to a race condition as the logs are flushing at the
conclusion of this test framework. If I fork off a process during
MesosSchedulerDriver::start() and use it to attach a debugger to the driver
process, I see the following at the SIGABRT:
{noformat}
(lldb) th li
Process 25383 stopped
thread #1: tid = 0x2e8f96, 0x7fff86ac84de
libsystem_kernel.dylib`mach_msg_trap + 10, queue = 'com.apple.main-thread'
thread #2: tid = 0x2e8fb9, 0x7fff86acd48a
libsystem_kernel.dylib`__semwait_signal + 10
thread #3: tid = 0x2e8fba, 0x7fff86ace232 libsystem_kernel.dylib`kevent64
+ 10, queue = 'com.apple.libdispatch-manager'
thread #4: tid = 0x2e8fbb, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #5: tid = 0x2e8fbc, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #6: tid = 0x2e8fbd, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #7: tid = 0x2e8fbe, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #8: tid = 0x2e8fbf, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #9: tid = 0x2e8fc0, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #10: tid = 0x2e8fc1, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #11: tid = 0x2e8fc2, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #12: tid = 0x2e8fc3, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #13: tid = 0x2e8fc4, 0x7fff8d8c4d62
libsystem_malloc.dylib`tiny_free_list_add_ptr + 108
thread #14: tid = 0x2e8fc5, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Reference Handler'
thread #15: tid = 0x2e8fc6, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Finalizer'
thread #18: tid = 0x2e8fcb, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Signal Dispatcher'
thread #19: tid = 0x2e8fcc, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C2 CompilerThread0'
thread #20: tid = 0x2e8fcd, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C2 CompilerThread1'
thread #21: tid = 0x2e8fce, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C2 CompilerThread2'
thread #22: tid = 0x2e8fcf, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: C1 CompilerThread3'
thread #23: tid = 0x2e8fd0, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10, name = 'Java: Service Thread'
thread #25: tid = 0x2e8fd2, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #26: tid = 0x2e8fd3, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #27: tid = 0x2e8fd4, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #28: tid = 0x2e8fd5, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #29: tid = 0x2e8fd6, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #30: tid = 0x2e8fd7, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
* thread #31: tid = 0x2e8fd8, 0x7fff86acd286
libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
thread #32: tid = 0x2e8fd9, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #33: tid = 0x2e8fda, 0x0001274f9fb0
libmesos-0.24.0.dylib`std::__1::atomic_flag*
synchronized_get_pointer(t=0x7ff1) at
synchronized.hpp:88
thread #34: tid = 0x2e8fdb, 0x7fff86acd136
libsystem_kernel.dylib`__psynch_cvwait + 10
(lldb) th b
* thread #31: tid = 0x2e8fd8, 0x7fff86acd286
libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
frame #0: 0x7fff86acd286 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x7fff87e8842f libsystem_pthread.dylib`pthread_kill + 90
* frame #2: 0x7fff8c237b53 libsystem_c.dylib`abort + 129
frame #3: 0x000129216010
libmesos-0.24.0.dylib`glog_internal_namespace_::Mutex::Lock(this=0x0001298b4c38)
+ 64 at mutex.h:248
frame #4: 0x000129215fc3
libmesos-0.24.0.dylib`glog_internal_namespace_::MutexLock::MutexLock(this=0x00012d945cc8,
mu=0x0001298b4c38) + 35 at mutex.h:288
frame #5: 0x00012921285d
libmesos-0.24.0.dylib`glog_internal_namespace_::MutexLock::MutexLock(this=0x00012d945cc8,
mu=0x0001298b4c38) + 29 at mutex.h:288
frame #6: 0x000129202f17
libmesos-0.24.0.dylib`google::LogMessage::Flush(this=0x00012d945e40) + 279
at logging.cc:1280
frame #7: 0x000129202db8
libmesos-0.24.0.dylib`google::LogMessage::~LogMessage(this=0x00012d945e40)
+ 24 at loggin
[jira] [Comment Edited] (MESOS-2860) Create the basic infrastructure to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14607993#comment-14607993
]
Isabel Jimenez edited comment on MESOS-2860 at 8/4/15 11:37 PM:
submitted:
https://reviews.apache.org/r/36040/
https://reviews.apache.org/r/36360/
https://reviews.apache.org/r/36328/
https://reviews.apache.org/r/35934/
https://reviews.apache.org/r/35939/
https://reviews.apache.org/r/36073/
https://reviews.apache.org/r/36072/
reviewable:
https://reviews.apache.org/r/36402/
https://reviews.apache.org/r/37097
https://reviews.apache.org/r/36624/
https://reviews.apache.org/r/36040/
discarded or split:
https://reviews.apache.org/r/36217/
https://reviews.apache.org/r/36037/
was (Author: ijimenez):
submitted:
https://reviews.apache.org/r/36040/
https://reviews.apache.org/r/36360/
https://reviews.apache.org/r/36328/
https://reviews.apache.org/r/35934/
https://reviews.apache.org/r/35939/
https://reviews.apache.org/r/36073/
https://reviews.apache.org/r/36072/
reviewable:
https://reviews.apache.org/r/36402/
https://reviews.apache.org/r/36624/
https://reviews.apache.org/r/36040/
discarded or split:
https://reviews.apache.org/r/36217/
https://reviews.apache.org/r/36037/
> Create the basic infrastructure to handle /call endpoint
>
>
> Key: MESOS-2860
> URL: https://issues.apache.org/jira/browse/MESOS-2860
> Project: Mesos
> Issue Type: Story
> Components: master
>Reporter: Marco Massenzio
>Assignee: Isabel Jimenez
> Labels: mesosphere
>
> This is the first basic step in ensuring the basic {{/call}} functionality:
> processing a
> {noformat}
> POST /call
> {noformat}
> and returning:
> - {{202}} if all goes well;
> - {{401}} if not authorized; and
> - {{403}} if the request is malformed.
> We'll get more sophisticated as the work progressed (eg, supporting {{415}}
> if the content-type is not of the right kind).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3205) No need to checkpoint container root filesystem path.
Jie Yu created MESOS-3205: - Summary: No need to checkpoint container root filesystem path. Key: MESOS-3205 URL: https://issues.apache.org/jira/browse/MESOS-3205 Project: Mesos Issue Type: Task Reporter: Jie Yu Given the design discussed in [MESOS-3004|https://issues.apache.org/jira/browse/MESOS-3004], one container might have multiple provisioned root filesystems. Only checkpointing the root filesystem for ContainerInfo::image does not make sense. Also, we realized that checkpointing container root filesystem path is not necessary because each provisioner should be able to destroy root filesystems for a given container based on a canonical directory layout (e.g., //xxx). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3204) PortMappingIsolatorProcess shell script can silently fail
[ https://issues.apache.org/jira/browse/MESOS-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654606#comment-14654606 ] Paul Brett commented on MESOS-3204: --- https://reviews.apache.org/r/37106 > PortMappingIsolatorProcess shell script can silently fail > - > > Key: MESOS-3204 > URL: https://issues.apache.org/jira/browse/MESOS-3204 > Project: Mesos > Issue Type: Bug > Components: isolation >Affects Versions: 0.24.0 >Reporter: Paul Brett >Assignee: Paul Brett > > PortMappingIsolatorProcess::scripts generates a shell script to configure the > target environment but does not set the shell '-e' flag. Hence errors > generated by the script will be silently ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2860) Create the basic infrastructure to handle /call endpoint
[
https://issues.apache.org/jira/browse/MESOS-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654619#comment-14654619
]
Anand Mazumdar commented on MESOS-2860:
---
{code}
commit 8e4d1c6e4fd1be2fe05db045f034b84bf19e04af
Author: Anand Mazumdar
Date: Tue Aug 4 13:16:58 2015 -0700
Added /call parsing and validation.
Review: https://reviews.apache.org/r/36720
{code}
> Create the basic infrastructure to handle /call endpoint
>
>
> Key: MESOS-2860
> URL: https://issues.apache.org/jira/browse/MESOS-2860
> Project: Mesos
> Issue Type: Story
> Components: master
>Reporter: Marco Massenzio
>Assignee: Isabel Jimenez
> Labels: mesosphere
>
> This is the first basic step in ensuring the basic {{/call}} functionality:
> processing a
> {noformat}
> POST /call
> {noformat}
> and returning:
> - {{202}} if all goes well;
> - {{401}} if not authorized; and
> - {{403}} if the request is malformed.
> We'll get more sophisticated as the work progressed (eg, supporting {{415}}
> if the content-type is not of the right kind).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3070) Master CHECK failure if a framework uses duplicated task id.
[
https://issues.apache.org/jira/browse/MESOS-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Klaus Ma updated MESOS-3070:
Shepherd: Vinod Kone
> Master CHECK failure if a framework uses duplicated task id.
>
>
> Key: MESOS-3070
> URL: https://issues.apache.org/jira/browse/MESOS-3070
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 0.22.1
>Reporter: Jie Yu
>Assignee: Klaus Ma
>
> We observed this in one of our testing cluster.
> One framework (under development) keeps launching tasks using the same
> task_id. We don't expect the master to crash even if the framework is not
> doing what it's supposed to do. However, under a series of events, this could
> happen and keeps crashing the master.
> 1) frameworkA launches task 'task_id_1' on slaveA
> 2) master fails over
> 3) slaveA has not re-registered yet
> 4) frameworkA re-registered and launches task 'task_id_1' on slaveB
> 5) slaveA re-registering and add task "task_id_1' to frameworkA
> 6) CHECK failure in addTask
> {noformat}
> I0716 21:52:50.759305 28805 master.hpp:159] Adding task 'task_id_1' with
> resources cpus(*):4; mem(*):32768 on slave
> 20150417-232509-1735470090-5050-48870-S25 (hostname)
> ...
> ...
> F0716 21:52:50.760136 28805 master.hpp:362] Check failed:
> !tasks.contains(task->task_id()) Duplicate task 'task_id_1' of framework
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3201) Libev handle_async can deadlock with run_in_event_loop
[
https://issues.apache.org/jira/browse/MESOS-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654649#comment-14654649
]
Benjamin Mahler commented on MESOS-3201:
Just to be clear, is it possible to trigger this deadlock currently?
> Libev handle_async can deadlock with run_in_event_loop
> --
>
> Key: MESOS-3201
> URL: https://issues.apache.org/jira/browse/MESOS-3201
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 0.23.0
>Reporter: Joris Van Remoortere
>Assignee: Joris Van Remoortere
> Labels: libprocess, mesosphere
>
> Due to the arbitrary nature of the functions that are executed in
> handle_async, invoking them under the (A) {{watchers_mutex}} can lead to
> deadlocks if (B) is acquired before calling {{run_in_event_loop}} and (B) is
> also acquired within the arbitrary function.
> {code}
> ==82679== Thread #10: lock order "0x60774F8 before 0x60768C0" violated
> ==82679==
> ==82679== Observed (incorrect) order is: acquisition of lock at 0x60768C0
> ==82679==at 0x4C32145: pthread_mutex_lock (in
> /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==82679==by 0x692C9B: __gthread_mutex_lock(pthread_mutex_t*)
> (gthr-default.h:748)
> ==82679==by 0x6950BF: std::mutex::lock() (mutex:134)
> ==82679==by 0x696219: Synchronized
> synchronize(std::mutex*)::{lambda(std::mutex*)#1}::operator()(std::mutex*)
> const (synchronized.hpp:58)
> ==82679==by 0x696238: Synchronized
> synchronize(std::mutex*)::{lambda(std::mutex*)#1}::_FUN(std::mutex*)
> (synchronized.hpp:58)
> ==82679==by 0x6984CF: Synchronized::Synchronized(std::mutex*,
> void (*)(std::mutex*), void (*)(std::mutex*)) (synchronized.hpp:35)
> ==82679==by 0x6962DE: Synchronized
> synchronize(std::mutex*) (synchronized.hpp:60)
> ==82679==by 0x728FE1: process::handle_async(ev_loop*, ev_async*, int)
> (libev.cpp:48)
> ==82679==by 0x761384: ev_invoke_pending (ev.c:2994)
> ==82679==by 0x7643C4: ev_run (ev.c:3394)
> ==82679==by 0x728E37: ev_loop (ev.h:826)
> ==82679==by 0x729469: process::EventLoop::run() (libev.cpp:135)
> ==82679==
> ==82679== followed by a later acquisition of lock at 0x60774F8
> ==82679==at 0x4C32145: pthread_mutex_lock (in
> /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
> ==82679==by 0x4C6F9D: __gthread_mutex_lock(pthread_mutex_t*)
> (gthr-default.h:748)
> ==82679==by 0x4C6FED: __gthread_recursive_mutex_lock(pthread_mutex_t*)
> (gthr-default.h:810)
> ==82679==by 0x4F5D3D: std::recursive_mutex::lock() (mutex:175)
> ==82679==by 0x516513: Synchronized
> synchronize(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::operator()(std::recursive_mutex*)
> const (synchronized.hpp:58)
> ==82679==by 0x516532: Synchronized
> synchronize(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::_FUN(std::recursive_mutex*)
> (synchronized.hpp:58)
> ==82679==by 0x52E619:
> Synchronized::Synchronized(std::recursive_mutex*, void
> (*)(std::recursive_mutex*), void (*)(std::recursive_mutex*))
> (synchronized.hpp:35)
> ==82679==by 0x5165D4: Synchronized
> synchronize(std::recursive_mutex*) (synchronized.hpp:60)
> ==82679==by 0x6BF4E1: process::ProcessManager::use(process::UPID const&)
> (process.cpp:2127)
> ==82679==by 0x6C2B8C: process::ProcessManager::terminate(process::UPID
> const&, bool, process::ProcessBase*) (process.cpp:2604)
> ==82679==by 0x6C6C3C: process::terminate(process::UPID const&, bool)
> (process.cpp:3107)
> ==82679==by 0x692B65: process::Latch::trigger() (latch.cpp:53)
> {code}
> This was introduced in
> https://github.com/apache/mesos/commit/849fc4d361e40062073324153ba97e98e294fdf2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3206) python tools do not obey PYTHON_VERSION
James Peach created MESOS-3206:
--
Summary: python tools do not obey PYTHON_VERSION
Key: MESOS-3206
URL: https://issues.apache.org/jira/browse/MESOS-3206
Project: Mesos
Issue Type: Bug
Components: python api
Affects Versions: 0.23.0
Reporter: James Peach
Priority: Minor
As per {{configure}}:
{code}
Some influential environment variables:
...
PYTHON_VERSION
The installed Python version to use, for example '2.3'. This
string will be appended to the Python interpreter canonical
name.
{code}
However, {{mesos-ps}} and other python tools install with the shebang line of
{{/usr/bin/env python}} even if you set {{PYTHON_VERSION}} to something else.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3206) python tools do not obey PYTHON_VERSION
[
https://issues.apache.org/jira/browse/MESOS-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654797#comment-14654797
]
James Peach commented on MESOS-3206:
Linked to other python installation bugs.
> python tools do not obey PYTHON_VERSION
> ---
>
> Key: MESOS-3206
> URL: https://issues.apache.org/jira/browse/MESOS-3206
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Affects Versions: 0.23.0
>Reporter: James Peach
>Priority: Minor
>
> As per {{configure}}:
> {code}
> Some influential environment variables:
> ...
> PYTHON_VERSION
> The installed Python version to use, for example '2.3'. This
> string will be appended to the Python interpreter canonical
> name.
> {code}
> However, {{mesos-ps}} and other python tools install with the shebang line of
> {{/usr/bin/env python}} even if you set {{PYTHON_VERSION}} to something else.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3206) python tools do not obey PYTHON_VERSION
[
https://issues.apache.org/jira/browse/MESOS-3206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654798#comment-14654798
]
James Peach commented on MESOS-3206:
Really link to other python install bugs this time.
> python tools do not obey PYTHON_VERSION
> ---
>
> Key: MESOS-3206
> URL: https://issues.apache.org/jira/browse/MESOS-3206
> Project: Mesos
> Issue Type: Bug
> Components: python api
>Affects Versions: 0.23.0
>Reporter: James Peach
>Priority: Minor
>
> As per {{configure}}:
> {code}
> Some influential environment variables:
> ...
> PYTHON_VERSION
> The installed Python version to use, for example '2.3'. This
> string will be appended to the Python interpreter canonical
> name.
> {code}
> However, {{mesos-ps}} and other python tools install with the shebang line of
> {{/usr/bin/env python}} even if you set {{PYTHON_VERSION}} to something else.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3202) Avoid frameworks starving in DRF allocator.
[ https://issues.apache.org/jira/browse/MESOS-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654897#comment-14654897 ] Joerg Schad commented on MESOS-3202: Short Update: After various discussions with [~alex-mesos] and [~benjaminhindman] we hope to avoid such situations using Quota (MESOS-1791). > Avoid frameworks starving in DRF allocator. > --- > > Key: MESOS-3202 > URL: https://issues.apache.org/jira/browse/MESOS-3202 > Project: Mesos > Issue Type: Bug >Reporter: Joerg Schad > > We currently run into issues with the DRF scheduler that frameworks do not > receive offers (see https://github.com/mesosphere/marathon/issues/1931 for > details). > Imagine that we have 10 frameworks and unallocated resources from a single > slave. > Allocation interval is 1 sec, and refuse_seconds (i.e. the time for which a > declined resource is filtered) is 3 sec across all frameworks. > Allocator offers resources to framework 1 (according to DRF) which declines > the offer immediately. > In the next allocation interval framework 1 is skipped due to the declined > offer before. Hence the next framework 2 is offered the resources, which it > also declines. > The same procedure in the next allocation interval (with framework 3). > In the next allocation interval the refuse_seconds for framework 1 are over, > and as it still has the lowest DRF share it gets the resource offered again, > which it again declines. And the cycle begins again > Framework 4 (which is actually waiting for this resource) is never offered > this resource. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
