[jira] [Created] (MESOS-1722) Wrong attributes separator in slave --help
Nikita Vetoshkin created MESOS-1722: --- Summary: Wrong attributes separator in slave --help Key: MESOS-1722 URL: https://issues.apache.org/jira/browse/MESOS-1722 Project: Mesos Issue Type: Bug Components: slave Reporter: Nikita Vetoshkin Assignee: Nikita Vetoshkin Priority: Trivial {{mesos-slave --help}} says ',' should be used as attributes separator {noformat} --attributes=VALUE Attributes of machine, in the form: rack:2 or 'rack:2,u:1' {noformat} But that doesn't work and according to sources ({{src/common/attributes.cpp}} string is tokenized by ';'. Thus help text should be trivially fixed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1718) Command executor can overcommit the slave.
[ https://issues.apache.org/jira/browse/MESOS-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-1718: -- Assignee: (was: Vinod Kone) > Command executor can overcommit the slave. > -- > > Key: MESOS-1718 > URL: https://issues.apache.org/jira/browse/MESOS-1718 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler > > Currently we give a small amount of resources to the command executor, in > addition to resources used by the command task: > https://github.com/apache/mesos/blob/0.20.0-rc1/src/slave/slave.cpp#L2448 > {code: title=} > ExecutorInfo Slave::getExecutorInfo( > const FrameworkID& frameworkId, > const TaskInfo& task) > { > ... > // Add an allowance for the command executor. This does lead to a > // small overcommit of resources. > executor.mutable_resources()->MergeFrom( > Resources::parse( > "cpus:" + stringify(DEFAULT_EXECUTOR_CPUS) + ";" + > "mem:" + stringify(DEFAULT_EXECUTOR_MEM.megabytes())).get()); > ... > } > {code} > This leads to an overcommit of the slave. Ideally, for command tasks we can > "transfer" all of the task resources to the executor at the slave / isolation > level. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MESOS-1722) Wrong attributes separator in slave --help
[ https://issues.apache.org/jira/browse/MESOS-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone resolved MESOS-1722. --- Resolution: Fixed Fix Version/s: 0.21.0 commit 419011065cd0b1e2f521a3f22e9df3ff5eb7bc36 Author: Nikita Vetoshkin Date: Tue Aug 19 10:40:23 2014 -0700 Fixed help string for "--attributes" flag. Review: https://reviews.apache.org/r/24840 > Wrong attributes separator in slave --help > -- > > Key: MESOS-1722 > URL: https://issues.apache.org/jira/browse/MESOS-1722 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Nikita Vetoshkin >Assignee: Nikita Vetoshkin >Priority: Trivial > Fix For: 0.21.0 > > > {{mesos-slave --help}} says ',' should be used as attributes separator > {noformat} > --attributes=VALUE Attributes of machine, in the > form: > rack:2 or 'rack:2,u:1' > {noformat} > But that doesn't work and according to sources ({{src/common/attributes.cpp}} > string is tokenized by ';'. Thus help text should be trivially fixed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1695) The stats.json endpoint on the slave exposes "registered" as a string.
[ https://issues.apache.org/jira/browse/MESOS-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-1695: -- Shepherd: Benjamin Mahler Assignee: Vinod Kone Story Points: 1 > The stats.json endpoint on the slave exposes "registered" as a string. > -- > > Key: MESOS-1695 > URL: https://issues.apache.org/jira/browse/MESOS-1695 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler >Assignee: Vinod Kone >Priority: Minor > Labels: newbie > > The slave is currently exposing a string value for the "registered" > statistic, this should be a number: > {code} > slave:5051/stats.json > { > "recovery_errors": 0, > "registered": "1", > "slave/executors_registering": 0, > ... > } > {code} > Should be a pretty straightforward fix, looks like this first originated back > in 2013: > {code} > commit b8291304e1523eb67ea8dc5f195cdb0d8e7d8348 > Author: Vinod Kone > Date: Wed Jul 3 12:37:36 2013 -0700 > Added a "registered" key/value pair to slave's stats.json. > Review: https://reviews.apache.org/r/12256 > diff --git a/src/slave/http.cpp b/src/slave/http.cpp > index dc2955f..dd51516 100644 > --- a/src/slave/http.cpp > +++ b/src/slave/http.cpp > @@ -281,6 +281,8 @@ Future Slave::Http::stats(const Request& > request) >object.values["lost_tasks"] = slave.stats.tasks[TASK_LOST]; >object.values["valid_status_updates"] = slave.stats.validStatusUpdates; >object.values["invalid_status_updates"] = slave.stats.invalidStatusUpdates; > + object.values["registered"] = slave.master ? "1" : "0"; > + >return OK(object, request.query.get("jsonp")); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1715) The slave does not send pending tasks / executors during re-registration.
[ https://issues.apache.org/jira/browse/MESOS-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler updated MESOS-1715: --- Summary: The slave does not send pending tasks / executors during re-registration. (was: The slave does not send pending tasks during re-registration.) > The slave does not send pending tasks / executors during re-registration. > - > > Key: MESOS-1715 > URL: https://issues.apache.org/jira/browse/MESOS-1715 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler >Assignee: Benjamin Mahler > > In what looks like an oversight, the pending tasks in the slave > (Framework::pending) are not sent in the re-registration message. > This can lead to spurious TASK_LOST notifications being generated by the > master when it falsely thinks the tasks are not present on the slave. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1715) The slave does not send pending tasks / executors during re-registration.
[ https://issues.apache.org/jira/browse/MESOS-1715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler updated MESOS-1715: --- Description: In what looks like an oversight, the pending tasks and executors in the slave (Framework::pending) are not sent in the re-registration message. For tasks, this can lead to spurious TASK_LOST notifications being generated by the master when it falsely thinks the tasks are not present on the slave. For executors, this can lead to under-accounting in the master, causing an overcommit on the slave. was: In what looks like an oversight, the pending tasks in the slave (Framework::pending) are not sent in the re-registration message. This can lead to spurious TASK_LOST notifications being generated by the master when it falsely thinks the tasks are not present on the slave. > The slave does not send pending tasks / executors during re-registration. > - > > Key: MESOS-1715 > URL: https://issues.apache.org/jira/browse/MESOS-1715 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler >Assignee: Benjamin Mahler > > In what looks like an oversight, the pending tasks and executors in the slave > (Framework::pending) are not sent in the re-registration message. > For tasks, this can lead to spurious TASK_LOST notifications being generated > by the master when it falsely thinks the tasks are not present on the slave. > For executors, this can lead to under-accounting in the master, causing an > overcommit on the slave. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MESOS-1695) The stats.json endpoint on the slave exposes "registered" as a string.
[ https://issues.apache.org/jira/browse/MESOS-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone resolved MESOS-1695. --- Resolution: Fixed Fix Version/s: 0.21.0 commit d7212c9f6b43cb74c50833c197e3775fea89209b Author: Vinod Kone Date: Tue Aug 19 11:15:25 2014 -0700 Fixed "registered" value in slave's /stats.json. > The stats.json endpoint on the slave exposes "registered" as a string. > -- > > Key: MESOS-1695 > URL: https://issues.apache.org/jira/browse/MESOS-1695 > Project: Mesos > Issue Type: Bug > Components: slave >Reporter: Benjamin Mahler >Assignee: Vinod Kone >Priority: Minor > Labels: newbie > Fix For: 0.21.0 > > > The slave is currently exposing a string value for the "registered" > statistic, this should be a number: > {code} > slave:5051/stats.json > { > "recovery_errors": 0, > "registered": "1", > "slave/executors_registering": 0, > ... > } > {code} > Should be a pretty straightforward fix, looks like this first originated back > in 2013: > {code} > commit b8291304e1523eb67ea8dc5f195cdb0d8e7d8348 > Author: Vinod Kone > Date: Wed Jul 3 12:37:36 2013 -0700 > Added a "registered" key/value pair to slave's stats.json. > Review: https://reviews.apache.org/r/12256 > diff --git a/src/slave/http.cpp b/src/slave/http.cpp > index dc2955f..dd51516 100644 > --- a/src/slave/http.cpp > +++ b/src/slave/http.cpp > @@ -281,6 +281,8 @@ Future Slave::Http::stats(const Request& > request) >object.values["lost_tasks"] = slave.stats.tasks[TASK_LOST]; >object.values["valid_status_updates"] = slave.stats.validStatusUpdates; >object.values["invalid_status_updates"] = slave.stats.invalidStatusUpdates; > + object.values["registered"] = slave.master ? "1" : "0"; > + >return OK(object, request.query.get("jsonp")); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MESOS-1723) Replace "BoundedRateLimiter" with EventThrottler libprocess Process
Yan Xu created MESOS-1723: - Summary: Replace "BoundedRateLimiter" with EventThrottler libprocess Process Key: MESOS-1723 URL: https://issues.apache.org/jira/browse/MESOS-1723 Project: Mesos Issue Type: Improvement Affects Versions: 0.20.0 Reporter: Yan Xu Priority: Minor To help enforce per-principal capacity for framework limiting, Master implements a nested struct [BoundedRateLimiter|https://github.com/apache/mesos/blob/0.20.0-rc2/src/master/master.hpp#L772] that keeps track of the capacity for the principal and the current number of outstanding messages. Master implements all the accounting logic ([e.g.|https://github.com/apache/mesos/blob/0.20.0-rc2/src/master/master.cpp#L879]) and makes sure the struct is used correctly. The capacity enforcing logic will be cleaner and better encapsulated by if we replace BoundedRateLimiter with an EventThrottler libprocess Process and move all the aforementioned logic inside. Note that the reason for not implementing the capacity inside [RateLimiter|https://github.com/apache/mesos/blob/0.20.0-rc2/3rdparty/libprocess/include/process/limiter.hpp] is that it is not aware of who's calling it and we have different rules for MessageEvents and ExitedEvents w.r.t capacity (i.e. ExitedEvents are not subject to capacity). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MESOS-1691) Improvement of Docker image location failure message
[ https://issues.apache.org/jira/browse/MESOS-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen resolved MESOS-1691. - Resolution: Fixed > Improvement of Docker image location failure message > > > Key: MESOS-1691 > URL: https://issues.apache.org/jira/browse/MESOS-1691 > Project: Mesos > Issue Type: Improvement >Reporter: Julien Eid >Assignee: Timothy Chen >Priority: Trivial > > I know this is pretty minor but is a bit confusing for people new to using > the Docker containerizer. The error message for a malformed Docker image URL > is "No docker image found, skipping launch", which was pretty confusing to me > as I had kept specifying the image and it was saying there was none instead > of telling me the string was wrong. I think it would be better if it was > "Incorrect Docker image specified, Docker image URL's must start with > docker:///" > I wouldn't mind working on this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1016) Mesos CLI improvements
[ https://issues.apache.org/jira/browse/MESOS-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103099#comment-14103099 ] Chris Aniszczyk commented on MESOS-1016: [~davelester], since this was our Outreach Program for Women project this summer with [~ijimenez], should we close this JIRA out with the details of the project? > Mesos CLI improvements > -- > > Key: MESOS-1016 > URL: https://issues.apache.org/jira/browse/MESOS-1016 > Project: Mesos > Issue Type: Improvement > Components: cli >Reporter: Dave Lester >Assignee: Thomas Rampelberg > Labels: gsoc2014 > > Add new functionality to the Mesos CLI to make developers lives easier. > More-specific features should be added attached as sub-issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1016) Mesos CLI improvements
[ https://issues.apache.org/jira/browse/MESOS-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103260#comment-14103260 ] Benjamin Hindman commented on MESOS-1016: - [~caniszczyk], this isn't what [~ijimenez] ended up working on, so we should leave it open for [~thomasr] to resolve. > Mesos CLI improvements > -- > > Key: MESOS-1016 > URL: https://issues.apache.org/jira/browse/MESOS-1016 > Project: Mesos > Issue Type: Improvement > Components: cli >Reporter: Dave Lester >Assignee: Thomas Rampelberg > Labels: gsoc2014 > > Add new functionality to the Mesos CLI to make developers lives easier. > More-specific features should be added attached as sub-issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MESOS-1724) Can't include port in DockerInfo's image
Jay Buffington created MESOS-1724: - Summary: Can't include port in DockerInfo's image Key: MESOS-1724 URL: https://issues.apache.org/jira/browse/MESOS-1724 Project: Mesos Issue Type: Bug Components: containerization Reporter: Jay Buffington Priority: Minor The current git tree doesn't allow you to specify a docker image with multiple colons. It is valid that multiple colons would exist in a docker image. e.g. docker-registry.example.com:80/centos:6u5 >From >https://github.com/apache/mesos/blob/02a35ab213fb074f6c532075cada76f13eb9d552/src/slave/containerizer/docker.cpp#L441 {code} vector parts = strings::split(dockerInfo.image(), ":"); if (parts.size() > 2) { return Failure("Not expecting multiple ':' in image: " + dockerInfo.image()); } {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MESOS-1725) Docker logs aren't appended to sandbox/stderr when container fails to start
Jay Buffington created MESOS-1725: - Summary: Docker logs aren't appended to sandbox/stderr when container fails to start Key: MESOS-1725 URL: https://issues.apache.org/jira/browse/MESOS-1725 Project: Mesos Issue Type: Bug Reporter: Jay Buffington If the docker containerizer fails to launch a docker container (say, because of a bogus command) the logs will not be written to the sandbox's stderr. This is because the logic of DockerContainerizerProcess doesn't account for calling "docker logs" when it encounters an error. As an aside, I think there might also be a separate bug that is related where a docker container which runs successfully and exits quickly will result in the docker containerizer incorrectly reporting failure. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (MESOS-1725) Docker logs aren't appended to sandbox/stderr when container fails to start
[ https://issues.apache.org/jira/browse/MESOS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen reassigned MESOS-1725: --- Assignee: Timothy Chen > Docker logs aren't appended to sandbox/stderr when container fails to start > --- > > Key: MESOS-1725 > URL: https://issues.apache.org/jira/browse/MESOS-1725 > Project: Mesos > Issue Type: Bug >Reporter: Jay Buffington >Assignee: Timothy Chen > Labels: docker > > If the docker containerizer fails to launch a docker container (say, because > of a bogus command) the logs will not be written to the sandbox's stderr. > This is because the logic of DockerContainerizerProcess doesn't account for > calling "docker logs" when it encounters an error. > As an aside, I think there might also be a separate bug that is related where > a docker container which runs successfully and exits quickly will result in > the docker containerizer incorrectly reporting failure. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1725) Docker logs aren't appended to sandbox/stderr when container fails to start
[ https://issues.apache.org/jira/browse/MESOS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-1725: Labels: docker (was: ) > Docker logs aren't appended to sandbox/stderr when container fails to start > --- > > Key: MESOS-1725 > URL: https://issues.apache.org/jira/browse/MESOS-1725 > Project: Mesos > Issue Type: Bug >Reporter: Jay Buffington > Labels: docker > > If the docker containerizer fails to launch a docker container (say, because > of a bogus command) the logs will not be written to the sandbox's stderr. > This is because the logic of DockerContainerizerProcess doesn't account for > calling "docker logs" when it encounters an error. > As an aside, I think there might also be a separate bug that is related where > a docker container which runs successfully and exits quickly will result in > the docker containerizer incorrectly reporting failure. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1725) Docker logs aren't appended to sandbox/stderr when container fails to start
[ https://issues.apache.org/jira/browse/MESOS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103367#comment-14103367 ] Timothy Chen commented on MESOS-1725: - I'll look into this, but in the other case you mention where it runs successfully we won't report any error as docker logs and docker wait all should run succesfully even when the container stops. > Docker logs aren't appended to sandbox/stderr when container fails to start > --- > > Key: MESOS-1725 > URL: https://issues.apache.org/jira/browse/MESOS-1725 > Project: Mesos > Issue Type: Bug >Reporter: Jay Buffington > Labels: docker > > If the docker containerizer fails to launch a docker container (say, because > of a bogus command) the logs will not be written to the sandbox's stderr. > This is because the logic of DockerContainerizerProcess doesn't account for > calling "docker logs" when it encounters an error. > As an aside, I think there might also be a separate bug that is related where > a docker container which runs successfully and exits quickly will result in > the docker containerizer incorrectly reporting failure. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MESOS-1724) Can't include port in DockerInfo's image
[ https://issues.apache.org/jira/browse/MESOS-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-1724: Labels: docker (was: ) > Can't include port in DockerInfo's image > > > Key: MESOS-1724 > URL: https://issues.apache.org/jira/browse/MESOS-1724 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Jay Buffington >Assignee: Timothy Chen >Priority: Minor > Labels: docker > > The current git tree doesn't allow you to specify a docker image with > multiple colons. It is valid that multiple colons would exist in a docker > image. e.g. docker-registry.example.com:80/centos:6u5 > From > https://github.com/apache/mesos/blob/02a35ab213fb074f6c532075cada76f13eb9d552/src/slave/containerizer/docker.cpp#L441 > {code} > vector parts = strings::split(dockerInfo.image(), ":"); > if (parts.size() > 2) { > return Failure("Not expecting multiple ':' in image: " + >dockerInfo.image()); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1724) Can't include port in DockerInfo's image
[ https://issues.apache.org/jira/browse/MESOS-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103369#comment-14103369 ] Timothy Chen commented on MESOS-1724: - I'm not able to create a docker image with colon in the name though: 2014/08/20 04:55:39 Invalid repository name (test:1234), only [a-z0-9-_.] are allowed > Can't include port in DockerInfo's image > > > Key: MESOS-1724 > URL: https://issues.apache.org/jira/browse/MESOS-1724 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Jay Buffington >Priority: Minor > Labels: docker > > The current git tree doesn't allow you to specify a docker image with > multiple colons. It is valid that multiple colons would exist in a docker > image. e.g. docker-registry.example.com:80/centos:6u5 > From > https://github.com/apache/mesos/blob/02a35ab213fb074f6c532075cada76f13eb9d552/src/slave/containerizer/docker.cpp#L441 > {code} > vector parts = strings::split(dockerInfo.image(), ":"); > if (parts.size() > 2) { > return Failure("Not expecting multiple ':' in image: " + >dockerInfo.image()); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (MESOS-1724) Can't include port in DockerInfo's image
[ https://issues.apache.org/jira/browse/MESOS-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen reassigned MESOS-1724: --- Assignee: Timothy Chen > Can't include port in DockerInfo's image > > > Key: MESOS-1724 > URL: https://issues.apache.org/jira/browse/MESOS-1724 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Jay Buffington >Assignee: Timothy Chen >Priority: Minor > Labels: docker > > The current git tree doesn't allow you to specify a docker image with > multiple colons. It is valid that multiple colons would exist in a docker > image. e.g. docker-registry.example.com:80/centos:6u5 > From > https://github.com/apache/mesos/blob/02a35ab213fb074f6c532075cada76f13eb9d552/src/slave/containerizer/docker.cpp#L441 > {code} > vector parts = strings::split(dockerInfo.image(), ":"); > if (parts.size() > 2) { > return Failure("Not expecting multiple ':' in image: " + >dockerInfo.image()); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1724) Can't include port in DockerInfo's image
[ https://issues.apache.org/jira/browse/MESOS-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103402#comment-14103402 ] Jay Buffington commented on MESOS-1724: --- I think the only possible second colon would be if you use a private registry and specify a port, which is valid. > Can't include port in DockerInfo's image > > > Key: MESOS-1724 > URL: https://issues.apache.org/jira/browse/MESOS-1724 > Project: Mesos > Issue Type: Bug > Components: containerization >Reporter: Jay Buffington >Assignee: Timothy Chen >Priority: Minor > Labels: docker > > The current git tree doesn't allow you to specify a docker image with > multiple colons. It is valid that multiple colons would exist in a docker > image. e.g. docker-registry.example.com:80/centos:6u5 > From > https://github.com/apache/mesos/blob/02a35ab213fb074f6c532075cada76f13eb9d552/src/slave/containerizer/docker.cpp#L441 > {code} > vector parts = strings::split(dockerInfo.image(), ":"); > if (parts.size() > 2) { > return Failure("Not expecting multiple ':' in image: " + >dockerInfo.image()); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)