[jira] [Commented] (MESOS-1762) Avoid docker pull on each container run
[ https://issues.apache.org/jira/browse/MESOS-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133491#comment-14133491 ] Bhuvan Arumugam commented on MESOS-1762: here's the revised patch which [~tnachen] had posted to avoid docker pull if it's already present: https://reviews.apache.org/r/25523/ > Avoid docker pull on each container run > --- > > Key: MESOS-1762 > URL: https://issues.apache.org/jira/browse/MESOS-1762 > Project: Mesos > Issue Type: Improvement >Reporter: Timothy Chen >Assignee: Timothy Chen > > Currently the docker containerizer does a docker pull on each run, and this > has several downsides: > 1. Not able to run local images > 2. Require to contact registry server on each run, therefore docker run > becomes unavailable if the registry server is down. Also has scalability > limits. > We want to avoid doing a pull everytime. The downside ofcourse is that images > without explicit tags (:latest) will not get the most updated version, but > this is the same behavior when calling docker run on the cli. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1621) Docker run networking should be configurable and support bridge network
[
https://issues.apache.org/jira/browse/MESOS-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133464#comment-14133464
]
Bhuvan Arumugam commented on MESOS-1621:
Adding support for {{--net=bridge}} is a feature, deserved to be part of
0.21.0. Considering the patch will go through few more iterations of review.
This need not be part of minor bug fix release 0.20.1.
If no one disagree, I'll move it to 0.21.0.
> Docker run networking should be configurable and support bridge network
> ---
>
> Key: MESOS-1621
> URL: https://issues.apache.org/jira/browse/MESOS-1621
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: Timothy Chen
>Assignee: Timothy Chen
> Labels: Docker
>
> Currently to easily support running executors in Docker image, we hardcode
> --net=host into Docker run so slave and executor and reuse the same mechanism
> to communicate, which is to pass the slave IP/PORT for the framework to
> respond with it's own hostname and port information back to setup the tunnel.
> We want to see how to abstract this or even get rid of host networking
> altogether if we have a good way to not rely on it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1741) mesos-slave shouldn't fail if dockerd is down
[
https://issues.apache.org/jira/browse/MESOS-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133456#comment-14133456
]
Bhuvan Arumugam commented on MESOS-1741:
As a user, i would expect to bring up mesos-slave with other containerizer if
dockerd is down. This wouldn't affect non-docker tasks.
> mesos-slave shouldn't fail if dockerd is down
> -
>
> Key: MESOS-1741
> URL: https://issues.apache.org/jira/browse/MESOS-1741
> Project: Mesos
> Issue Type: Bug
> Components: slave
>Affects Versions: 0.20.1
>Reporter: Bhuvan Arumugam
> Fix For: 0.20.1
>
>
> When using {{--containerizers=docker,mesos}} for mesos-slave, it fail to come
> up if dockerd is not running. It use {{docker version}} to figure out docker
> access. The {{docker version}} exit 1 if dockerd is not running.
> mesos-slave should launch with other containerzer (mesos), if dockerd is down.
> {code}
> I0827 21:33:23.953763 19448 logging.cpp:142] INFO level logging started!
> I0827 21:33:23.954180 19448 main.cpp:126] Build: 2014-08-21 21:26:28 by
> jenkins
> I0827 21:33:23.954190 19448 main.cpp:128] Version: 0.21.0
> I0827 21:33:23.954196 19448 main.cpp:135] Git SHA:
> 70784a9f234b2902d6fee11298365d9b08756313
> Failed to create a containerizer: Could not create DockerContainerizer:
> Failed to execute 'docker version': exited with status exited with status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1791) Introduce Master / Offer Resource Reservations
[ https://issues.apache.org/jira/browse/MESOS-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler updated MESOS-1791: --- Issue Type: Epic (was: Story) > Introduce Master / Offer Resource Reservations > -- > > Key: MESOS-1791 > URL: https://issues.apache.org/jira/browse/MESOS-1791 > Project: Mesos > Issue Type: Epic >Reporter: Tom Arnfeld > > Currently Mesos supports the ability to reserve resources (for a given role) > on a per-slave basis, as introduced in MESOS-505. This allows you to almost > statically partition off a set of resources on a set of machines, to > guarantee certain types of frameworks get some resources. > This is very useful, though it is also very useful to be able to control > these reservations through the master (instead of per-slave) for when I don't > care which nodes I get on, as long as I get X cpu and Y RAM, or Z sets of > (X,Y). > I'm not sure what structure this could take, but apparently it has already > been discussed. Would this be a CLI flag? Could there be a (authenticated) > web interface to control these reservations? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1791) Introduce Master / Offer Resource Reservations
[ https://issues.apache.org/jira/browse/MESOS-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler updated MESOS-1791: --- Epic Name: Offer Reservations > Introduce Master / Offer Resource Reservations > -- > > Key: MESOS-1791 > URL: https://issues.apache.org/jira/browse/MESOS-1791 > Project: Mesos > Issue Type: Epic >Reporter: Tom Arnfeld > > Currently Mesos supports the ability to reserve resources (for a given role) > on a per-slave basis, as introduced in MESOS-505. This allows you to almost > statically partition off a set of resources on a set of machines, to > guarantee certain types of frameworks get some resources. > This is very useful, though it is also very useful to be able to control > these reservations through the master (instead of per-slave) for when I don't > care which nodes I get on, as long as I get X cpu and Y RAM, or Z sets of > (X,Y). > I'm not sure what structure this could take, but apparently it has already > been discussed. Would this be a CLI flag? Could there be a (authenticated) > web interface to control these reservations? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1791) Introduce Master / Offer Resource Reservations
[ https://issues.apache.org/jira/browse/MESOS-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler updated MESOS-1791: --- Affects Version/s: (was: 0.20.0) > Introduce Master / Offer Resource Reservations > -- > > Key: MESOS-1791 > URL: https://issues.apache.org/jira/browse/MESOS-1791 > Project: Mesos > Issue Type: Epic >Reporter: Tom Arnfeld > > Currently Mesos supports the ability to reserve resources (for a given role) > on a per-slave basis, as introduced in MESOS-505. This allows you to almost > statically partition off a set of resources on a set of machines, to > guarantee certain types of frameworks get some resources. > This is very useful, though it is also very useful to be able to control > these reservations through the master (instead of per-slave) for when I don't > care which nodes I get on, as long as I get X cpu and Y RAM, or Z sets of > (X,Y). > I'm not sure what structure this could take, but apparently it has already > been discussed. Would this be a CLI flag? Could there be a (authenticated) > web interface to control these reservations? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Domański updated MESOS-1794:
--
Description:
This divides {{Try net::download(const std::string& url, const
std::string& path)}} into three overloads:
# {{Try net::download(const std::string& url, FILE* file)}} downloads a
file and sends it to FILE stream
# {{Try net::download(const std::string& url, int fdout)}} opens a file
stream based on a file descriptor and calls 1)
# {{Try net::download(const std::string& url, const std::string& path)}}
opens a file at _path_ and passes the descriptor to 2), effectively working as
before this change
This will allow to download into any file descriptor, such as a pipe, e.g.
directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
Under similar rationale it's necessary to implement {{hadoop fs -cat}} into a
file descriptor, but I'm unable to deploy HDFS for testing.
was:
This divides {{Try net::download(const std::string& url, const
std::string& path)}} into three overloads:
# {{Try net::download(const std::string& url, FILE* file)}} downloads a
file and sends it to FILE stream
# {{Try net::download(const std::string& url, int fdout)}} opens a file
stream based on a file descriptor and calls 1)
# {{Try net::download(const std::string& url, const std::string& path)}}
opens a file at _path_ and passes the descriptor to 2), effectively working as
before this change
This will allow to download into any file descriptor, such as a pipe, e.g.
directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
Under similiar rationale it's necessary to implement {{hadoop fs -cat}} into a
file descriptor, but I'm unable to deploy HDFS for testing.
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.20.0
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, patch, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
> Under similar rationale it's necessary to implement {{hadoop fs -cat}} into a
> file descriptor, but I'm unable to deploy HDFS for testing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Domański updated MESOS-1794:
--
Labels: features patch performance (was: features performance)
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.20.0
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, patch, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
> Under similar rationale it's necessary to implement {{hadoop fs -cat}} into a
> file descriptor, but I'm unable to deploy HDFS for testing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Domański updated MESOS-1794:
--
Description:
This divides {{Try net::download(const std::string& url, const
std::string& path)}} into three overloads:
# {{Try net::download(const std::string& url, FILE* file)}} downloads a
file and sends it to FILE stream
# {{Try net::download(const std::string& url, int fdout)}} opens a file
stream based on a file descriptor and calls 1)
# {{Try net::download(const std::string& url, const std::string& path)}}
opens a file at _path_ and passes the descriptor to 2), effectively working as
before this change
This will allow to download into any file descriptor, such as a pipe, e.g.
directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
Under similiar rationale it's necessary to implement {{hadoop fs -cat}} into a
file descriptor, but I'm unable to deploy HDFS for testing.
was:
This divides {{Try net::download(const std::string& url, const
std::string& path)}} into three overloads:
# {{Try net::download(const std::string& url, FILE* file)}} downloads a
file and sends it to FILE stream
# {{Try net::download(const std::string& url, int fdout)}} opens a file
stream based on a file descriptor and calls 1)
# {{Try net::download(const std::string& url, const std::string& path)}}
opens a file at _path_ and passes the descriptor to 2), effectively working as
before this change
This will allow to download into any file descriptor, such as a pipe, e.g.
directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.20.0
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
> Under similiar rationale it's necessary to implement {{hadoop fs -cat}} into
> a file descriptor, but I'm unable to deploy HDFS for testing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Domański updated MESOS-1794:
--
Affects Version/s: 0.20.0
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.20.0
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamil Domański updated MESOS-1794:
--
Target Version/s: 1.0.0, 0.21.0 (was: 0.21.0)
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.20.0
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1792) add os::shell with pluggable IO descriptors
[ https://issues.apache.org/jira/browse/MESOS-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Domański updated MESOS-1792: -- Target Version/s: 1.0.0, 0.21.0 (was: 0.21.0) > add os::shell with pluggable IO descriptors > --- > > Key: MESOS-1792 > URL: https://issues.apache.org/jira/browse/MESOS-1792 > Project: Mesos > Issue Type: Improvement > Components: stout >Affects Versions: 0.20.0 >Reporter: Kamil Domański >Assignee: Kamil Domański > Labels: features, patch, performance > > Add an overload of os::shell that allows to run a command with stdin and/or > stdout substituted by file descriptors passed as parameters. > This will allow to pipe a stream of data in and out of a process e.g. > directly from download to extraction, as proposed by [~bernd-mesos] in > MESOS-1667. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1794) allow CURL download into a pipe
[
https://issues.apache.org/jira/browse/MESOS-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133297#comment-14133297
]
Kamil Domański commented on MESOS-1794:
---
https://reviews.apache.org/r/25623/
> allow CURL download into a pipe
> ---
>
> Key: MESOS-1794
> URL: https://issues.apache.org/jira/browse/MESOS-1794
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Reporter: Kamil Domański
>Assignee: Kamil Domański
> Labels: features, performance
>
> This divides {{Try net::download(const std::string& url, const
> std::string& path)}} into three overloads:
> # {{Try net::download(const std::string& url, FILE* file)}} downloads a
> file and sends it to FILE stream
> # {{Try net::download(const std::string& url, int fdout)}} opens a file
> stream based on a file descriptor and calls 1)
> # {{Try net::download(const std::string& url, const std::string& path)}}
> opens a file at _path_ and passes the descriptor to 2), effectively working
> as before this change
> This will allow to download into any file descriptor, such as a pipe, e.g.
> directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1794) allow CURL download into a pipe
Kamil Domański created MESOS-1794:
-
Summary: allow CURL download into a pipe
Key: MESOS-1794
URL: https://issues.apache.org/jira/browse/MESOS-1794
Project: Mesos
Issue Type: Improvement
Components: stout
Reporter: Kamil Domański
Assignee: Kamil Domański
This divides {{Try net::download(const std::string& url, const
std::string& path)}} into three overloads:
# {{Try net::download(const std::string& url, FILE* file)}} downloads a
file and sends it to FILE stream
# {{Try net::download(const std::string& url, int fdout)}} opens a file
stream based on a file descriptor and calls 1)
# {{Try net::download(const std::string& url, const std::string& path)}}
opens a file at _path_ and passes the descriptor to 2), effectively working as
before this change
This will allow to download into any file descriptor, such as a pipe, e.g.
directly to extraction, as proposed by [~bernd-mesos] in MESOS-1667.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-1788) Add c++11 feature whitelist to style guide
[
https://issues.apache.org/jira/browse/MESOS-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rukletsov updated MESOS-1788:
---
Issue Type: Technical task (was: Documentation)
Parent: MESOS-1793
> Add c++11 feature whitelist to style guide
> --
>
> Key: MESOS-1788
> URL: https://issues.apache.org/jira/browse/MESOS-1788
> Project: Mesos
> Issue Type: Technical task
> Components: documentation
>Reporter: Dominic Hamon
>Priority: Minor
>
> We now support a subset of C++11 as checked by the {{configure}} script. It
> would be useful to have documentation in the style guide regarding the white
> list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1778) Provide an option to validate flag value in stout/flags.
[
https://issues.apache.org/jira/browse/MESOS-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133233#comment-14133233
]
Alexander Rukletsov commented on MESOS-1778:
I think it makes sense to embed it into the {{FlagsBase::add()}}, so we have
flag definition, default value and validation checks in the same place.
> Provide an option to validate flag value in stout/flags.
> -
>
> Key: MESOS-1778
> URL: https://issues.apache.org/jira/browse/MESOS-1778
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Reporter: Alexander Rukletsov
>Priority: Minor
>
> Currently we can provide the default value for a flag, but cannot check if
> the flag is set to a reasonable value and, e.g., issue a warning. Passing an
> optional lambda checker to {{FlagBase::add()}} can be a possible solution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-1793) Update mesos style guide with C++11 and naming notes.
Alexander Rukletsov created MESOS-1793: -- Summary: Update mesos style guide with C++11 and naming notes. Key: MESOS-1793 URL: https://issues.apache.org/jira/browse/MESOS-1793 Project: Mesos Issue Type: Documentation Components: documentation Reporter: Alexander Rukletsov Assignee: Alexander Rukletsov Priority: Minor Explicitly prohibit the use of namespace aliases. Add a note about allowed C++11 features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
