[jira] [Created] (MESOS-9803) Memory leak caused by an infinite chain of futures in `UriDiskProfileAdaptor`.
Chun-Hung Hsiao created MESOS-9803:
--
Summary: Memory leak caused by an infinite chain of futures in
`UriDiskProfileAdaptor`.
Key: MESOS-9803
URL: https://issues.apache.org/jira/browse/MESOS-9803
Project: Mesos
Issue Type: Bug
Components: storage
Affects Versions: 1.8.0, 1.7.1, 1.7.0, 1.6.2, 1.6.1, 1.5.3, 1.5.2
Reporter: Chun-Hung Hsiao
Assignee: Chun-Hung Hsiao
Before MESOS-8906, {{UriDiskProfileAdaptor}} only update its promise for
watchers if the polled profile matrix becomes larger in size, and this prevents
the following code in the {{watch}} function from creating an infinite chain of
futures when the profile matrix keeps the same:
https://github.com/apache/mesos/blob/fa410f2fb8efb988590f4da2d4cfffbb2ce70637/src/resource_provider/storage/uri_disk_profile_adaptor.cpp#L159-L160
However, the patch of MESOS-8906 removes the size check in the {{notify}}
function to allow profile selectors to be updated. As a result, once the watch
function is called, the returned future will be chained with a new promise
every time a poll is made, hence creating a memory leak.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-9793) Implement UPDATE_FRAMEWORK call in V0 API
[ https://issues.apache.org/jira/browse/MESOS-9793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851205#comment-16851205 ] Andrei Sekretenko edited comment on MESOS-9793 at 5/29/19 6:51 PM: --- Implementation in the master and in the scheduler driver: [https://reviews.apache.org/r/70751/] [https://reviews.apache.org/r/70752/] Tests: [https://reviews.apache.org/r/70753/] [https://reviews.apache.org/r/70754/] [https://reviews.apache.org/r/70755/] was (Author: asekretenko): Implementation in the master and in the scheduler driver: [https://reviews.apache.org/r/70751/ ] [https://reviews.apache.org/r/70752/] Tests: [https://reviews.apache.org/r/70753/] [https://reviews.apache.org/r/70754/] https://reviews.apache.org/r/70755/ > Implement UPDATE_FRAMEWORK call in V0 API > - > > Key: MESOS-9793 > URL: https://issues.apache.org/jira/browse/MESOS-9793 > Project: Mesos > Issue Type: Task >Reporter: Andrei Sekretenko >Assignee: Andrei Sekretenko >Priority: Major > Labels: multitenancy, resource-management > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9793) Implement UPDATE_FRAMEWORK call in V0 API
[ https://issues.apache.org/jira/browse/MESOS-9793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851205#comment-16851205 ] Andrei Sekretenko commented on MESOS-9793: -- Implementation in the master and in the scheduler driver: [https://reviews.apache.org/r/70751/ ] [https://reviews.apache.org/r/70752/] Tests: [https://reviews.apache.org/r/70753/] [https://reviews.apache.org/r/70754/] https://reviews.apache.org/r/70755/ > Implement UPDATE_FRAMEWORK call in V0 API > - > > Key: MESOS-9793 > URL: https://issues.apache.org/jira/browse/MESOS-9793 > Project: Mesos > Issue Type: Task >Reporter: Andrei Sekretenko >Assignee: Andrei Sekretenko >Priority: Major > Labels: multitenancy, resource-management > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9770) Add no-new-privileges isolator.
[
https://issues.apache.org/jira/browse/MESOS-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851203#comment-16851203
]
Jacob Janco commented on MESOS-9770:
[https://reviews.apache.org/r/70757/]
> Add no-new-privileges isolator.
> ---
>
> Key: MESOS-9770
> URL: https://issues.apache.org/jira/browse/MESOS-9770
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: James Peach
>Assignee: Jacob Janco
>Priority: Major
>
> To give security-minded operators more defense in depth, add a {{linux/nnp}}
> isolator that sets the no-new-privileges bit before starting the executor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-9770) Add no-new-privileges isolator.
[
https://issues.apache.org/jira/browse/MESOS-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacob Janco reassigned MESOS-9770:
--
Assignee: Jacob Janco
> Add no-new-privileges isolator.
> ---
>
> Key: MESOS-9770
> URL: https://issues.apache.org/jira/browse/MESOS-9770
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: James Peach
>Assignee: Jacob Janco
>Priority: Major
>
> To give security-minded operators more defense in depth, add a {{linux/nnp}}
> isolator that sets the no-new-privileges bit before starting the executor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-9802) Remove quota role sorter in the allocator.
Meng Zhu created MESOS-9802: --- Summary: Remove quota role sorter in the allocator. Key: MESOS-9802 URL: https://issues.apache.org/jira/browse/MESOS-9802 Project: Mesos Issue Type: Improvement Components: allocation Reporter: Meng Zhu Assignee: Meng Zhu Remove the dedicated quota role sorter in favor of using the same sorting between satisfying guarantees and bursting above guarantees up to limits. This is tech debt from when a "quota role" was considered different from a "non-quota" role. However, they are the same, one just has a default quota. The only practical difference between quota role sorter and role sorter now is that quota role sorter ignores the revocable resources both in its total resource pool as well as role allocations. Thus when using DRF, it does not count revocable resources which is arguably the right behavior. By removing the quota sorter, we will have all roles sorted together. When using DRF, in the 1st quota guarantee allocation stage, its share calculation will also include revocable resources. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850626#comment-16850626
]
Felipe Alfaro Solana commented on MESOS-9800:
-
[^certificates2.tar.gz]
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9800) libarchive cannot extract tarfile due to UTF-8 encoding issues
[
https://issues.apache.org/jira/browse/MESOS-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850625#comment-16850625
]
Felipe Alfaro Solana commented on MESOS-9800:
-
I'm attaching the tar file that we the Marathon application depends on.
> libarchive cannot extract tarfile due to UTF-8 encoding issues
> --
>
> Key: MESOS-9800
> URL: https://issues.apache.org/jira/browse/MESOS-9800
> Project: Mesos
> Issue Type: Bug
> Components: fetcher
>Affects Versions: 1.7.2
> Environment: Mesos 1.7.2 and Marathon 1.4.3 running on top of Ubuntu
> 16.04.
>Reporter: Felipe Alfaro Solana
>Priority: Major
> Attachments: certificates2.tar.gz
>
>
> Starting with Mesos 1.7, the following change has been introduced:
> * [MESOS-8064] - Mesos now requires libarchive to programmatically decode
> .zip, .tar, .gzip, and other common file compression schemes. Version 3.3.2
> is bundled in Mesos.
> However, this version of libarchive which is used by the fetcher component in
> Mesos has problems in dealing with archive files (.tar and .zip) which
> contain UTF-8 characters. We run Marahton on top of Mesos, and one of our
> Marathon application relies on a .tar file which contains symlinks whose
> target contains certain UTF-8 characters (Turkish) or the symlink name itself
> contains UTF-8 characters. Mesos fetcher is unable to extract the archive and
> fails with the following error:
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.791250 6136 fetcher.cpp:613] EXIT with status 1: Failed to fetch
> '/tmp/certificates.tar.gz': Failed to extract archive
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0/certificates.tar.gz'
> to
> '/var/mesos/slaves/10c35371-f690-4d40-8b9e-30ffd04405fb-S6/frameworks/ff2993eb-987f-47b0-b3af-fb8b49ab0470-/executors/test-nginx.fe01a0c0-8135-11e9-a160-02427a38aa03/runs/6a6e87e8-5eef-4e8e-8c00-3f081fa187b0':
> Failed to read archive header: Linkname can't be converted from UTF-8 to
> current locale.}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]:}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: End
> fetcher log for container 6a6e87e8-5eef-4e8e-8c00-3f081fa187b0}}
> {{May 28 10:47:30 t01m01.node.t01.dns.teralytics.net mesos-slave[4319]: E0528
> 10:47:30.846695 4343 fetcher.cpp:571] Failed to run mesos-fetcher: Failed to
> fetch all URIs for container '6a6e87e8-5eef-4e8e-8c00-3f081fa187b0': exited
> with status 1}}
> The same Marathon application works fine with Mesos 1.6 which does not use
> libarchive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
