[jira] [Commented] (MESOS-2827) Broken Doxygen rake target for generating website.
[ https://issues.apache.org/jira/browse/MESOS-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936467#comment-14936467 ] Joerg Schad commented on MESOS-2827: Hi, the change was that the doxyfile is now supposed to be called from the build dir and will then generate the output then into build/doxygen. As there was an effort to restructure the rake file (as it would not deal with images correctly) we were waiting for that change to happen. Hope this makes sense, Cheers Joerg > Broken Doxygen rake target for generating website. > -- > > Key: MESOS-2827 > URL: https://issues.apache.org/jira/browse/MESOS-2827 > Project: Mesos > Issue Type: Documentation >Reporter: Joerg Schad > > With the recent changes to the Doxygen file > (https://reviews.apache.org/r/34938/) which generate the doxygen documenation > into the build folder (and not the main docs folder) we need to update the > rakefile https://svn.apache.org/repos/asf/mesos/. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3555) mesos.cli brokern
Radoslaw Gruchalski created MESOS-3555: -- Summary: mesos.cli brokern Key: MESOS-3555 URL: https://issues.apache.org/jira/browse/MESOS-3555 Project: Mesos Issue Type: Bug Reporter: Radoslaw Gruchalski -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3556) mesos.cli broken in 0.24.0+
Radoslaw Gruchalski created MESOS-3556:
--
Summary: mesos.cli broken in 0.24.0+
Key: MESOS-3556
URL: https://issues.apache.org/jira/browse/MESOS-3556
Project: Mesos
Issue Type: Bug
Components: cli
Affects Versions: 0.24.1, 0.24.0
Reporter: Radoslaw Gruchalski
The issue was initially reported on the mailing list:
http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
The format of the master data stored in zookeeper has changed but the mesos.cli
does not reflect these changes causing tools like {{mesos-tail}} and
{{mesos-ps}} to fail.
Example error from {{mesos-tail}}:
{noformat}
mesos-master ~$ mesos tail -f -n 50 service
Traceback (most recent call last):
File "/usr/local/bin/mesos-tail", line 11, in
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
55, in main
args.task, args.file, fail=(not args.follow)):
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line 27,
in files
tlist = MASTER.tasks(fltr)
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 174,
in tasks
self._task_list(active_only
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 153,
in _task_list
*[util.merge(x, *keys) for x in self.frameworks(active_only)])
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 185,
in frameworks
return util.merge(self.state, *keys)
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58, in
__get__
value = self.fget(inst)
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 123,
in state
return self.fetch("/master/state.json").json()
File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
in fetch
return requests.get(urlparse.urljoin(self.host, url), **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
request
response = session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 451,
in request
prep = self.prepare_request(req)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 382,
in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
in prepare
self.prepare_url(url, params)
File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
in prepare_url
raise InvalidURL(*e.args)
requests.exceptions.InvalidURL: Failed to parse:
10.100.1.100:5050","port":5050,"version":"0.24.1"}
{noformat}
The problem exists in
https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
The code should be along the lines of:
{noformat}
try:
parsed = json.loads(val)
return parsed["address"]["ip"] + ":" +
str(parsed["address"]["port"])
except Exception:
return val.split("@")[-1]
{noformat}
This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3556) mesos.cli broken in 0.24.0+
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-3556:
---
Labels: mesosphere (was: )
> mesos.cli broken in 0.24.0+
> ---
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-3556) mesos.cli broken in 0.24.0+
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio reassigned MESOS-3556:
--
Assignee: Marco Massenzio
> mesos.cli broken in 0.24.0+
> ---
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-3556:
---
Summary: mesos.cli broken in 0.24.x (was: mesos.cli broken in 0.24.0+)
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-3556:
---
Shepherd: (was: Marco Massenzio)
Sprint: Mesosphere Sprint 20
Story Points: 1
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936708#comment-14936708 ] Ian Babrou commented on MESOS-2613: --- Any feedback on this? Orphaned volumes overflow disks on our slaves. > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Priority: Minor > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3552) Check failed: result.cpus() == cpus() && result.mem() == mem() && result.disk() == disk() && result.ports() == ports()
[
https://issues.apache.org/jira/browse/MESOS-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936979#comment-14936979
]
James Peach commented on MESOS-3552:
FWIW, Glog has {{CHECK_DOUBLE_EQ}} for this.
> Check failed: result.cpus() == cpus() && result.mem() == mem() &&
> result.disk() == disk() && result.ports() == ports()
> ---
>
> Key: MESOS-3552
> URL: https://issues.apache.org/jira/browse/MESOS-3552
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 0.24.1
>Reporter: Mandeep Chadha
>Assignee: Mandeep Chadha
>
> result.cpus() == cpus() check is failing due to ( double == double )
> comparison problem.
> Root Cause :
> Framework requested 0.1 cpu reservation for the first task. So far so good.
> Next Reserve operation — lead to double operations resulting in following
> double values :
> results.cpus() : 23.9964472863211995 cpus() : 24
> And the check ( result.cpus() == cpus() ) failed.
> The double arithmetic operations caused results.cpus() value to be :
> 23.9964472863211995 and hence ( 23.9964472863211995
> == 24 ) failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3552) Check failed: result.cpus() == cpus() && result.mem() == mem() && result.disk() == disk() && result.ports() == ports()
[
https://issues.apache.org/jira/browse/MESOS-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937026#comment-14937026
]
Mandeep Chadha commented on MESOS-3552:
---
One more example of ( double == double) check - though doesn't result in crash
:)
Util function will be useful but still have to be enforced. Maybe it is time to
wrap double primitive into Double ( primitive wrapper class) ? That way we
don't have to chase down every (double == double) check.
https://github.com/apache/mesos/blob/master/src/master/allocator/sorter/drf/sorter.cpp#L34
bool DRFComparator::operator()(const Client& client1, const Client& client2)
{
if (client1.share == client2.share) {
if (client1.allocations == client2.allocations) {
return client1.name < client2.name;
}
return client1.allocations < client2.allocations;
}
return client1.share < client2.share;
}
Where share is double :
https://github.com/apache/mesos/blob/master/src/master/allocator/sorter/drf/sorter.hpp#L43
struct Client
{
Client(const std::string& _name, double _share, uint64_t _allocations)
: name(_name), share(_share), allocations(_allocations) {}
std::string name;
double share;
---
As far as this bug, we can use :
"
The CHECK_DOUBLE_EQ macro checks the equality of two floating point values,
accepting a small error margin. CHECK_NEAR accepts a third floating point
argument, which specifies the acceptable error margin.
"
> Check failed: result.cpus() == cpus() && result.mem() == mem() &&
> result.disk() == disk() && result.ports() == ports()
> ---
>
> Key: MESOS-3552
> URL: https://issues.apache.org/jira/browse/MESOS-3552
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 0.24.1
>Reporter: Mandeep Chadha
>Assignee: Mandeep Chadha
>
> result.cpus() == cpus() check is failing due to ( double == double )
> comparison problem.
> Root Cause :
> Framework requested 0.1 cpu reservation for the first task. So far so good.
> Next Reserve operation — lead to double operations resulting in following
> double values :
> results.cpus() : 23.9964472863211995 cpus() : 24
> And the check ( result.cpus() == cpus() ) failed.
> The double arithmetic operations caused results.cpus() value to be :
> 23.9964472863211995 and hence ( 23.9964472863211995
> == 24 ) failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3552) Check failed: result.cpus() == cpus() && result.mem() == mem() && result.disk() == disk() && result.ports() == ports()
[ https://issues.apache.org/jira/browse/MESOS-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937031#comment-14937031 ] Mandeep Chadha commented on MESOS-3552: --- Framework : Reserve 0.1 CPU Launch task Reserve 0.1 CPU -- Crash I will add a unit test. > Check failed: result.cpus() == cpus() && result.mem() == mem() && > result.disk() == disk() && result.ports() == ports() > --- > > Key: MESOS-3552 > URL: https://issues.apache.org/jira/browse/MESOS-3552 > Project: Mesos > Issue Type: Bug > Components: master >Affects Versions: 0.24.1 >Reporter: Mandeep Chadha >Assignee: Mandeep Chadha > > result.cpus() == cpus() check is failing due to ( double == double ) > comparison problem. > Root Cause : > Framework requested 0.1 cpu reservation for the first task. So far so good. > Next Reserve operation — lead to double operations resulting in following > double values : > results.cpus() : 23.9964472863211995 cpus() : 24 > And the check ( result.cpus() == cpus() ) failed. > The double arithmetic operations caused results.cpus() value to be : > 23.9964472863211995 and hence ( 23.9964472863211995 > == 24 ) failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3557) Interpretation of { "type": "NONE" } in access control lists
James Fisher created MESOS-3557:
---
Summary: Interpretation of { "type": "NONE" } in access control
lists
Key: MESOS-3557
URL: https://issues.apache.org/jira/browse/MESOS-3557
Project: Mesos
Issue Type: Documentation
Reporter: James Fisher
I am trying to understand the documentation for the authentication system. The
interpretation of the
{ "type": "NONE" }
string predicate is strange. In particular I want to know what the
interpretation of the following ACL should be:
{code}
{
"run_tasks": [
{
"principals": { "type": "NONE" },
"users": { "type": "NONE" }
}
]
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937442#comment-14937442
]
Niklas Quarfot Nielsen commented on MESOS-3556:
---
[~marco-mesos] Do you have/need a shepherd for this?
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3554) Allocator changes trigger large re-compiles
[ https://issues.apache.org/jira/browse/MESOS-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937445#comment-14937445 ] Niklas Quarfot Nielsen commented on MESOS-3554: --- Do you need a shepherd for this? > Allocator changes trigger large re-compiles > --- > > Key: MESOS-3554 > URL: https://issues.apache.org/jira/browse/MESOS-3554 > Project: Mesos > Issue Type: Improvement > Components: allocation >Reporter: Joris Van Remoortere >Assignee: Joris Van Remoortere > Labels: mesosphere > > Due to the templatized nature of the allocator, even small changes trigger > large recompiles of the code-base. This make iterating on changes expensive > for developers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1615) Create design document for Optimistic Offers
[ https://issues.apache.org/jira/browse/MESOS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen updated MESOS-1615: -- Shepherd: Joris Van Remoortere > Create design document for Optimistic Offers > > > Key: MESOS-1615 > URL: https://issues.apache.org/jira/browse/MESOS-1615 > Project: Mesos > Issue Type: Documentation >Reporter: Dominic Hamon >Assignee: Joseph Wu > Labels: mesosphere > > As a first step toward Optimistic Offers, take the description from the epic > and build an implementation design doc that can be shared for comments. > Note: the links to the working group notes and design doc are located in the > [JIRA Epic|MESOS-1607]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3515) Support Subscribe Call for HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937447#comment-14937447
]
Niklas Quarfot Nielsen commented on MESOS-3515:
---
Hi [~anandmazumdar] - have you found a shepherd for this?
> Support Subscribe Call for HTTP based Executors
> ---
>
> Key: MESOS-3515
> URL: https://issues.apache.org/jira/browse/MESOS-3515
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> We need to add a {{subscribe(...)}} method in {{src/slave/slave.cpp}} to
> introduce the ability for HTTP based executors to subscribe and then receive
> events on the persistent HTTP connection. Most of the functionality needed
> would be similar to {{Master::subscribe}} in {{src/master/master.cpp}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3515) Support Subscribe Call for HTTP based Executors
[
https://issues.apache.org/jira/browse/MESOS-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Mazumdar updated MESOS-3515:
--
Shepherd: Vinod Kone
> Support Subscribe Call for HTTP based Executors
> ---
>
> Key: MESOS-3515
> URL: https://issues.apache.org/jira/browse/MESOS-3515
> Project: Mesos
> Issue Type: Task
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
> Labels: mesosphere
>
> We need to add a {{subscribe(...)}} method in {{src/slave/slave.cpp}} to
> introduce the ability for HTTP based executors to subscribe and then receive
> events on the persistent HTTP connection. Most of the functionality needed
> would be similar to {{Master::subscribe}} in {{src/master/master.cpp}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3099) Validation of Docker Image Manifests from Docker Registry
[ https://issues.apache.org/jira/browse/MESOS-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-3099: -- Shepherd: Timothy Chen > Validation of Docker Image Manifests from Docker Registry > - > > Key: MESOS-3099 > URL: https://issues.apache.org/jira/browse/MESOS-3099 > Project: Mesos > Issue Type: Improvement >Reporter: Lily Chen >Assignee: Gilbert Song > Labels: mesosphere > > Docker image manifests pulled from remote Docker registries should be > verified against their signature digest before they are used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3468) Improve apply_reviews.sh script to apply chain of reviews
[ https://issues.apache.org/jira/browse/MESOS-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937452#comment-14937452 ] Adam B commented on MESOS-3468: --- [~vinodkone] Are you shepherding this? > Improve apply_reviews.sh script to apply chain of reviews > - > > Key: MESOS-3468 > URL: https://issues.apache.org/jira/browse/MESOS-3468 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: Artem Harutyunyan > Labels: mesosphere > > Currently the support/apply-review.sh script allows an user (typically > committer) to apply a single review on top the HEAD. Since Mesos contributors > typically submit a chain of reviews for a given issue it makes sense for the > script to apply the whole chain recursively. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3183) Documentation images do not load
[
https://issues.apache.org/jira/browse/MESOS-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937451#comment-14937451
]
Niklas Quarfot Nielsen commented on MESOS-3183:
---
[~davelester] Can you be the shepherd for this?
> Documentation images do not load
>
>
> Key: MESOS-3183
> URL: https://issues.apache.org/jira/browse/MESOS-3183
> Project: Mesos
> Issue Type: Documentation
> Components: documentation
>Affects Versions: 0.24.0
>Reporter: James Mulcahy
>Assignee: Joseph Wu
>Priority: Minor
> Labels: mesosphere
> Attachments: rake.patch
>
>
> Any images which are referenced from the generated docs ({{docs/*.md}}) do
> not show up on the website. For example:
> * [External
> Containerizer|http://mesos.apache.org/documentation/latest/external-containerizer/]
> * [Fetcher Cache
> Internals|http://mesos.apache.org/documentation/latest/fetcher-cache-internals/]
> * [Maintenance|http://mesos.apache.org/documentation/latest/maintenance/]
> *
> [Oversubscription|http://mesos.apache.org/documentation/latest/oversubscription/]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Mann reassigned MESOS-2613: Assignee: Greg Mann > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Assignee: Greg Mann >Priority: Minor > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-2613: Shepherd: Timothy Chen > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Assignee: Greg Mann >Priority: Minor > Labels: mesosphere > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937704#comment-14937704 ] Timothy Chen commented on MESOS-2613: - Sorry we haven't got to do this, I think this is worth fixing. > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Assignee: Greg Mann >Priority: Minor > Labels: mesosphere > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy Chen updated MESOS-2613: Labels: mesosphere (was: ) > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Assignee: Greg Mann >Priority: Minor > Labels: mesosphere > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3371) Implement process::subprocess on Windows
[ https://issues.apache.org/jira/browse/MESOS-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Clemmer reassigned MESOS-3371: --- Assignee: Alex Clemmer > Implement process::subprocess on Windows > > > Key: MESOS-3371 > URL: https://issues.apache.org/jira/browse/MESOS-3371 > Project: Mesos > Issue Type: Task > Components: libprocess >Reporter: Alex Clemmer >Assignee: Alex Clemmer > Labels: libprocess, mesosphere > > From a discussion with mpark we (IIRC) concluded that even on Windows we call > this a couple times. We need to (1) confirm, and (2) do it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2533) Support HTTP checks in Mesos health check program
[ https://issues.apache.org/jira/browse/MESOS-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938216#comment-14938216 ] Tomás Senart commented on MESOS-2533: - Any blockers on this one? With this in place, we'd only be missing the TCP checks to match Marathon's health-checking functionality which could then be delegated. > Support HTTP checks in Mesos health check program > - > > Key: MESOS-2533 > URL: https://issues.apache.org/jira/browse/MESOS-2533 > Project: Mesos > Issue Type: Bug >Reporter: Niklas Quarfot Nielsen >Assignee: haosdent > > Currently, only commands are supported but our health check protobuf enables > users to encode HTTP checks as well. We should wire up this in the health > check program or remove the http field from the protobuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-3409) Refactor the plain JSON parsing in the docker containerizer
[
https://issues.apache.org/jira/browse/MESOS-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Wu updated MESOS-3409:
-
Assignee: Gilbert Song (was: Joseph Wu)
> Refactor the plain JSON parsing in the docker containerizer
> ---
>
> Key: MESOS-3409
> URL: https://issues.apache.org/jira/browse/MESOS-3409
> Project: Mesos
> Issue Type: Improvement
> Components: docker
>Reporter: Joseph Wu
>Assignee: Gilbert Song
>Priority: Minor
>
> Two functions in the Docker-related code take a string and parse it to JSON:
> * {{Docker::Container::create}} in {{src/docker/docker.cpp}}
> * {{Token::create}} in
> {{src/slave/containerizer/provisioners/docker/token_manager.cpp}}
> This JSON is then validated (lots of if-elses) and used via the
> {{JSON::Value}} accessors. We could instead use a protobuf and the related
> Stout JSON->Protobuf conversion function.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2533) Support HTTP checks in Mesos health check program
[ https://issues.apache.org/jira/browse/MESOS-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938259#comment-14938259 ] Adam B commented on MESOS-2533: --- Sorry, this fell off my radar when I went on vacation. No blockers (besides my inattention). I will get back to reviewing it, and maybe [~haosd...@gmail.com] can rebase it. > Support HTTP checks in Mesos health check program > - > > Key: MESOS-2533 > URL: https://issues.apache.org/jira/browse/MESOS-2533 > Project: Mesos > Issue Type: Bug >Reporter: Niklas Quarfot Nielsen >Assignee: haosdent > > Currently, only commands are supported but our health check protobuf enables > users to encode HTTP checks as well. We should wire up this in the health > check program or remove the http field from the protobuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2972) Serialize Docker image spec as protobuf
[ https://issues.apache.org/jira/browse/MESOS-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938270#comment-14938270 ] Gilbert Song commented on MESOS-2972: - Marco, thanks for your concern. And yes, it can return errors if no corresponding message fields matched with JSON::Object fields, or it can easily cause segmentation faults because of incorrect JSON format. > Serialize Docker image spec as protobuf > --- > > Key: MESOS-2972 > URL: https://issues.apache.org/jira/browse/MESOS-2972 > Project: Mesos > Issue Type: Improvement >Reporter: Timothy Chen >Assignee: Gilbert Song > Labels: mesosphere > > The Docker image specification defines a schema for the metadata json that it > puts into each image. Currently the docker image provisioner needs to be able > to parse and understand this metadata json, and we should create a protobuf > equivelent schema so we can utilize the json to protobuf conversion to read > and validate the metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2972) Serialize Docker image spec as protobuf
[ https://issues.apache.org/jira/browse/MESOS-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938271#comment-14938271 ] Gilbert Song commented on MESOS-2972: - https://reviews.apache.org/r/38901/ > Serialize Docker image spec as protobuf > --- > > Key: MESOS-2972 > URL: https://issues.apache.org/jira/browse/MESOS-2972 > Project: Mesos > Issue Type: Improvement >Reporter: Timothy Chen >Assignee: Gilbert Song > Labels: mesosphere > > The Docker image specification defines a schema for the metadata json that it > puts into each image. Currently the docker image provisioner needs to be able > to parse and understand this metadata json, and we should create a protobuf > equivelent schema so we can utilize the json to protobuf conversion to read > and validate the metadata. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938275#comment-14938275
]
Marco Massenzio commented on MESOS-3556:
Great question - the way I see it, this is not in the `mesos` codebase, and
it's Python.
Suggestions?
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3558) Make the CommandExecutor use the Executor Library speaking HTTP
Anand Mazumdar created MESOS-3558:
-
Summary: Make the CommandExecutor use the Executor Library
speaking HTTP
Key: MESOS-3558
URL: https://issues.apache.org/jira/browse/MESOS-3558
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
Instead of using the {{MesosExecutorDriver}} , we should make the
{{CommandExecutor}} in {{src/launcher/executor.cpp}} use the new Executor HTTP
Library that we create in {{MESOS-3550}}.
This would act as a good validation of the {{HTTP API}} implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3559) Make the Command Scheduler use the HTTP Scheduler Library
Anand Mazumdar created MESOS-3559:
-
Summary: Make the Command Scheduler use the HTTP Scheduler Library
Key: MESOS-3559
URL: https://issues.apache.org/jira/browse/MESOS-3559
Project: Mesos
Issue Type: Task
Reporter: Anand Mazumdar
We should make the Command Scheduler in {{src/cli/executor.cpp}} use the
Scheduler Library {{src/scheduler/scheduler.cpp}} instead of the Scheduler
Driver.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3560) JSON-based credential files do not work correctly
Michael Park created MESOS-3560:
---
Summary: JSON-based credential files do not work correctly
Key: MESOS-3560
URL: https://issues.apache.org/jira/browse/MESOS-3560
Project: Mesos
Issue Type: Bug
Components: master
Reporter: Michael Park
Assignee: Isabel Jimenez
Specifying the following credentials file:
{code}
{
“credentials”: [
{
“principal”: “user”,
“secret”: “password”
}
]
}
{code}
Then hitting a master endpoint with:
{code}
curl -i -u “user:password” ...
{code}
Does not work. This is contrary to the text-based credentials file which works:
{code}
user password
{code}
Currently, the password in a JSON-based credentials file needs to be
base64-encoded in order for it to work:
{code}
{
“credentials”: [
{
“principal”: “user”,
“secret”: “cGFzc3dvcmQ=”
}
]
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3560) JSON-based credential files do not work correctly
[
https://issues.apache.org/jira/browse/MESOS-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Isabel Jimenez updated MESOS-3560:
--
Sprint: Mesosphere Sprint 20
Story Points: 1
Fix Version/s: 0.20.0
0.21.0
0.22.0
0.23.0
0.24.0
> JSON-based credential files do not work correctly
> -
>
> Key: MESOS-3560
> URL: https://issues.apache.org/jira/browse/MESOS-3560
> Project: Mesos
> Issue Type: Bug
> Components: master
>Reporter: Michael Park
>Assignee: Isabel Jimenez
> Labels: mesosphere
> Fix For: 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0
>
>
> Specifying the following credentials file:
> {code}
> {
> “credentials”: [
> {
> “principal”: “user”,
> “secret”: “password”
> }
> ]
> }
> {code}
> Then hitting a master endpoint with:
> {code}
> curl -i -u “user:password” ...
> {code}
> Does not work. This is contrary to the text-based credentials file which
> works:
> {code}
> user password
> {code}
> Currently, the password in a JSON-based credentials file needs to be
> base64-encoded in order for it to work:
> {code}
> {
> “credentials”: [
> {
> “principal”: “user”,
> “secret”: “cGFzc3dvcmQ=”
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3063) Add an example framework using dynamic reservation
[ https://issues.apache.org/jira/browse/MESOS-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Park updated MESOS-3063: Sprint: Mesosphere Sprint 20 > Add an example framework using dynamic reservation > -- > > Key: MESOS-3063 > URL: https://issues.apache.org/jira/browse/MESOS-3063 > Project: Mesos > Issue Type: Task >Reporter: Michael Park >Assignee: Klaus Ma > > An example framework using dynamic reservation should added to > # test dynamic reservations further, and > # to be used as a reference for those who want to use the dynamic reservation > feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938853#comment-14938853
]
Niklas Quarfot Nielsen commented on MESOS-3556:
---
While not a master at Python; I can help taking a look at it with you and help
land it
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2990) Task dropped into LOST state
[
https://issues.apache.org/jira/browse/MESOS-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938870#comment-14938870
]
Rohan Nahata commented on MESOS-2990:
-
[~ykrips] can you point me to how you actually went about fixing this... new to
mesos and have no clue as to what I'm doing
> Task dropped into LOST state
>
>
> Key: MESOS-2990
> URL: https://issues.apache.org/jira/browse/MESOS-2990
> Project: Mesos
> Issue Type: Bug
> Components: slave
>Affects Versions: 0.22.1
> Environment: RHEL 7.0 ppc64
> IBM JDK 1.7.0 SR 7
>Reporter: Jihun Kang
>
> Every time I ran "test-framework" command on the shell, mesos always failed
> to run each tasks. First task on this framework dropped into the *LOST*
> state, and another tasks also terminated.
> Following is the message from "test-framework".
> {code}
> # ./src/test-framework --master=10.10.14.72:5050
> I0706 17:24:44.202020 38486 sched.cpp:157] Version: 0.22.1
> I0706 17:24:44.210917 38523 sched.cpp:254] New master detected at
> master@10.10.14.72:5050
> I0706 17:24:44.212316 38523 sched.cpp:264] No credentials provided.
> Attempting to register without authentication
> I0706 17:24:44.215756 38529 sched.cpp:448] Framework registered with
> 20150706-154445-168431176-5050-19360-
> Registered!
> Received offer 20150706-154445-168431176-5050-19360-O0 with cpus(*):40;
> mem(*):60064; disk(*):46055; ports(*):[31000-32000]
> Launching task 0 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 1 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 2 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 3 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 4 using offer 20150706-154445-168431176-5050-19360-O0
> Task 0 is in state TASK_LOST
> Aborting because task 0 is in unexpected state TASK_LOST with reason 1 from
> source 1 with message 'Executor terminated'
> I0706 17:24:44.428568 38513 sched.cpp:1623] Asked to abort the driver
> I0706 17:24:44.428665 38513 sched.cpp:856] Aborting framework
> '20150706-154445-168431176-5050-19360-'
> I0706 17:24:44.428987 38486 sched.cpp:1589] Asked to stop the driver
> I0706 17:24:44.429121 38539 sched.cpp:831] Stopping framework
> '20150706-154445-168431176-5050-19360-'
> {code}
> Followings also got from the slave log.
> {code}
> I0706 17:24:44.225492 19452 slave.cpp:1144] Got assigned task 0 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.226763 19452 slave.cpp:1144] Got assigned task 1 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.227041 19452 slave.cpp:1144] Got assigned task 2 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.227252 19452 slave.cpp:1254] Launching task 0 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.238466 19452 slave.cpp:4208] Launching executor default of
> framework 20150706-154445-168431176-5050-19360- in work directory
> '/tmp/mesos/slaves/20150706-154445-168431176-5050-19360-S0/frameworks/20150706-154445-168431176-5050-19360-/executors/default/runs/d235751e-986c-44ae-a6c9-953814dac2f8'
> I0706 17:24:44.239434 19460 containerizer.cpp:484] Starting container
> 'd235751e-986c-44ae-a6c9-953814dac2f8' for executor 'default' of framework
> '20150706-154445-168431176-5050-19360-'
> I0706 17:24:44.239447 19452 slave.cpp:1401] Queuing task '0' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.239769 19452 slave.cpp:1144] Got assigned task 3 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240003 19452 slave.cpp:1254] Launching task 1 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240056 19452 slave.cpp:1401] Queuing task '1' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240100 19452 slave.cpp:1254] Launching task 2 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240146 19452 slave.cpp:1401] Queuing task '2' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240228 19452 slave.cpp:1144] Got assigned task 4 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240414 19452 slave.cpp:1254] Launching task 3 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240460 19452 slave.cpp:1401] Queuing task '3' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240501 19452 slave.cpp:1254] Launching task 4 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240592 19452 slave.cpp:1401] Queuing task '4' for executor
> default of framework '20150706-154445-168431176-5050-19
[jira] [Created] (MESOS-3561) Create a user doc for network isolation using per-container IP
Kapil Arya created MESOS-3561: - Summary: Create a user doc for network isolation using per-container IP Key: MESOS-3561 URL: https://issues.apache.org/jira/browse/MESOS-3561 Project: Mesos Issue Type: Bug Reporter: Kapil Arya Assignee: Kapil Arya Priority: Blocker -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2990) Task dropped into LOST state
[
https://issues.apache.org/jira/browse/MESOS-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938870#comment-14938870
]
Rohan Nahata edited comment on MESOS-2990 at 9/30/15 9:03 PM:
--
[~ykrips] can you point me to how you actually went about fixing this... new to
mesos and have no clue as to what I'm doing.
This is the error message that i got from the log that [~vinodkone] mentioned
error while loading shared libraries: libmesos-0.24.0.so: cannot open shared
object file: No such file or directory
was (Author: rohanahata):
[~ykrips] can you point me to how you actually went about fixing this... new to
mesos and have no clue as to what I'm doing
> Task dropped into LOST state
>
>
> Key: MESOS-2990
> URL: https://issues.apache.org/jira/browse/MESOS-2990
> Project: Mesos
> Issue Type: Bug
> Components: slave
>Affects Versions: 0.22.1
> Environment: RHEL 7.0 ppc64
> IBM JDK 1.7.0 SR 7
>Reporter: Jihun Kang
>
> Every time I ran "test-framework" command on the shell, mesos always failed
> to run each tasks. First task on this framework dropped into the *LOST*
> state, and another tasks also terminated.
> Following is the message from "test-framework".
> {code}
> # ./src/test-framework --master=10.10.14.72:5050
> I0706 17:24:44.202020 38486 sched.cpp:157] Version: 0.22.1
> I0706 17:24:44.210917 38523 sched.cpp:254] New master detected at
> master@10.10.14.72:5050
> I0706 17:24:44.212316 38523 sched.cpp:264] No credentials provided.
> Attempting to register without authentication
> I0706 17:24:44.215756 38529 sched.cpp:448] Framework registered with
> 20150706-154445-168431176-5050-19360-
> Registered!
> Received offer 20150706-154445-168431176-5050-19360-O0 with cpus(*):40;
> mem(*):60064; disk(*):46055; ports(*):[31000-32000]
> Launching task 0 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 1 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 2 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 3 using offer 20150706-154445-168431176-5050-19360-O0
> Launching task 4 using offer 20150706-154445-168431176-5050-19360-O0
> Task 0 is in state TASK_LOST
> Aborting because task 0 is in unexpected state TASK_LOST with reason 1 from
> source 1 with message 'Executor terminated'
> I0706 17:24:44.428568 38513 sched.cpp:1623] Asked to abort the driver
> I0706 17:24:44.428665 38513 sched.cpp:856] Aborting framework
> '20150706-154445-168431176-5050-19360-'
> I0706 17:24:44.428987 38486 sched.cpp:1589] Asked to stop the driver
> I0706 17:24:44.429121 38539 sched.cpp:831] Stopping framework
> '20150706-154445-168431176-5050-19360-'
> {code}
> Followings also got from the slave log.
> {code}
> I0706 17:24:44.225492 19452 slave.cpp:1144] Got assigned task 0 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.226763 19452 slave.cpp:1144] Got assigned task 1 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.227041 19452 slave.cpp:1144] Got assigned task 2 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.227252 19452 slave.cpp:1254] Launching task 0 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.238466 19452 slave.cpp:4208] Launching executor default of
> framework 20150706-154445-168431176-5050-19360- in work directory
> '/tmp/mesos/slaves/20150706-154445-168431176-5050-19360-S0/frameworks/20150706-154445-168431176-5050-19360-/executors/default/runs/d235751e-986c-44ae-a6c9-953814dac2f8'
> I0706 17:24:44.239434 19460 containerizer.cpp:484] Starting container
> 'd235751e-986c-44ae-a6c9-953814dac2f8' for executor 'default' of framework
> '20150706-154445-168431176-5050-19360-'
> I0706 17:24:44.239447 19452 slave.cpp:1401] Queuing task '0' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.239769 19452 slave.cpp:1144] Got assigned task 3 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240003 19452 slave.cpp:1254] Launching task 1 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240056 19452 slave.cpp:1401] Queuing task '1' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240100 19452 slave.cpp:1254] Launching task 2 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240146 19452 slave.cpp:1401] Queuing task '2' for executor
> default of framework '20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240228 19452 slave.cpp:1144] Got assigned task 4 for framework
> 20150706-154445-168431176-5050-19360-
> I0706 17:24:44.240414 19452 slave.cpp:1254] Launching task 3 for framework
> 20150706-154445-168431176-5050-19360-
[jira] [Updated] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marco Massenzio updated MESOS-3556:
---
Shepherd: Niklas Quarfot Nielsen
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3556) mesos.cli broken in 0.24.x
[
https://issues.apache.org/jira/browse/MESOS-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938903#comment-14938903
]
Marco Massenzio commented on MESOS-3556:
That works thanks!
(no mastery needed here, [~radekg] has already pretty much handed over the
solution :)
> mesos.cli broken in 0.24.x
> --
>
> Key: MESOS-3556
> URL: https://issues.apache.org/jira/browse/MESOS-3556
> Project: Mesos
> Issue Type: Bug
> Components: cli
>Affects Versions: 0.24.0, 0.24.1
>Reporter: Radoslaw Gruchalski
>Assignee: Marco Massenzio
> Labels: mesosphere
>
> The issue was initially reported on the mailing list:
> http://www.mail-archive.com/user@mesos.apache.org/msg04670.html
> The format of the master data stored in zookeeper has changed but the
> mesos.cli does not reflect these changes causing tools like {{mesos-tail}}
> and {{mesos-ps}} to fail.
> Example error from {{mesos-tail}}:
> {noformat}
> mesos-master ~$ mesos tail -f -n 50 service
> Traceback (most recent call last):
> File "/usr/local/bin/mesos-tail", line 11, in
> sys.exit(main())
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cli.py", line 61, in
> wrapper
> return fn(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cmds/tail.py", line
> 55, in main
> args.task, args.file, fail=(not args.follow)):
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/cluster.py", line
> 27,
> in files
> tlist = MASTER.tasks(fltr)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 174,
> in tasks
> self._task_list(active_only
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 153,
> in _task_list
> *[util.merge(x, *keys) for x in self.frameworks(active_only)])
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 185,
> in frameworks
> return util.merge(self.state, *keys)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/util.py", line 58,
> in
> __get__
> value = self.fget(inst)
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line
> 123,
> in state
> return self.fetch("/master/state.json").json()
> File "/usr/local/lib/python2.7/dist-packages/mesos/cli/master.py", line 64,
> in fetch
> return requests.get(urlparse.urljoin(self.host, url), **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in
> get
> return request('get', url, params=params, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in
> request
> response = session.request(method=method, url=url, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 451,
> in request
> prep = self.prepare_request(req)
> File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line
> 382,
> in prepare_request
> hooks=merge_hooks(request.hooks, self.hooks),
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 304,
> in prepare
> self.prepare_url(url, params)
> File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 357,
> in prepare_url
> raise InvalidURL(*e.args)
> requests.exceptions.InvalidURL: Failed to parse:
> 10.100.1.100:5050","port":5050,"version":"0.24.1"}
> {noformat}
> The problem exists in
> https://github.com/mesosphere/mesos-cli/blob/master/mesos/cli/master.py#L107.
> The code should be along the lines of:
> {noformat}
> try:
> parsed = json.loads(val)
> return parsed["address"]["ip"] + ":" +
> str(parsed["address"]["port"])
> except Exception:
> return val.split("@")[-1]
> {noformat}
> This causes the master address to come back correctly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Whitehead updated MESOS-3562:
-
Attachment: tcpdump.log
app.log
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> tcpdump.log contains a tcp dump between the mesos master and my client.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to this framework generated by Mesos
> are streamed on this connection. Master encodes each Event in RecordIO
> format, i.e., string representation of length of the event followed by JSON
> or binary Protobuf (possibly compressed) encoded event.
> {quote}
> There is no specification I've been able to find online that actually
> explains this format. The only reference I can find to it is some sample go
> code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
Ben Whitehead created MESOS-3562:
Summary: Anomalous bytes in stream from HTTPI Api
Key: MESOS-3562
URL: https://issues.apache.org/jira/browse/MESOS-3562
Project: Mesos
Issue Type: Bug
Components: HTTP API
Affects Versions: 0.24.0
Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3 14:37:06
UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
Mesos 0.24.0
gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
Reporter: Ben Whitehead
Priority: Blocker
Attachments: app.log, tcpdump.log
When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
some anomalous bytes contained in the chunked stream that appear to be causing
problems when I attempting to integrate.
Attached are two log files. app.log represents my application trying to connect
to mesos using RxNetty. Netty has been configured to log all data it
sends/receives over the wire this can be seen in the byte blocks in the log.
tcpdump.log contains a tcp dump between the mesos master and my client.
The client is constructing a protobuf in java for the subscribe call
{code:java}
final Call subscribeCall = Call.newBuilder()
.setType(Call.Type.SUBSCRIBE)
.setSubscribe(
Call.Subscribe.newBuilder()
.setFrameworkInfo(
Protos.FrameworkInfo.newBuilder()
.setUser("bill")
.setName("testing_this_shit_out")
.build()
)
)
.build();
{code}
lient sends the protobuf to mesos with the following request headers:
{code}
POST /api/v1/scheduler HTTP/1.1
Content-Type: application/x-protobuf
Accept: application/json
Content-Length: 35
Host: localhost:5050
User-Agent: RxNetty Client
{code}
The body is then serialized via protobuf and sent.
The response from the mesos master has the following headers:
{code}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Wed, 30 Sep 2015 21:07:16 GMT
Content-Type: application/json
{code}
followed by
{code}
\r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
{code}
The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
doesn't appear to be attached to anything. {{104}} is the correct length of the
Subscribe events JSON.
What is this extra number and why is it there?
This is not the first time confusion has come up related to the wire format for
the event stream from the new http api see
[this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
message from the mailing list.
In the [Design
Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
there is a statement that said
{quote}
All subsequent events that are relevant to this framework generated by Mesos
are streamed on this connection. Master encodes each Event in RecordIO format,
i.e., string representation of length of the event followed by JSON or binary
Protobuf (possibly compressed) encoded event.
{quote}
There is no specification I've been able to find online that actually explains
this format. The only reference I can find to it is some sample go code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Whitehead updated MESOS-3562:
-
Description:
When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
some anomalous bytes contained in the chunked stream that appear to be causing
problems when I attempting to integrate.
Attached are two log files. app.log represents my application trying to connect
to mesos using RxNetty. Netty has been configured to log all data it
sends/receives over the wire this can be seen in the byte blocks in the log.
The client is constructing a protobuf in java for the subscribe call
{code:java}
final Call subscribeCall = Call.newBuilder()
.setType(Call.Type.SUBSCRIBE)
.setSubscribe(
Call.Subscribe.newBuilder()
.setFrameworkInfo(
Protos.FrameworkInfo.newBuilder()
.setUser("bill")
.setName("testing_this_shit_out")
.build()
)
)
.build();
{code}
lient sends the protobuf to mesos with the following request headers:
{code}
POST /api/v1/scheduler HTTP/1.1
Content-Type: application/x-protobuf
Accept: application/json
Content-Length: 35
Host: localhost:5050
User-Agent: RxNetty Client
{code}
The body is then serialized via protobuf and sent.
The response from the mesos master has the following headers:
{code}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Wed, 30 Sep 2015 21:07:16 GMT
Content-Type: application/json
{code}
followed by
{code}
\r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
{code}
The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
doesn't appear to be attached to anything. {{104}} is the correct length of the
Subscribe events JSON.
What is this extra number and why is it there?
This is not the first time confusion has come up related to the wire format for
the event stream from the new http api see
[this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
message from the mailing list.
In the [Design
Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
there is a statement that said
{quote}
All subsequent events that are relevant to this framework generated by Mesos
are streamed on this connection. Master encodes each Event in RecordIO format,
i.e., string representation of length of the event followed by JSON or binary
Protobuf (possibly compressed) encoded event.
{quote}
There is no specification I've been able to find online that actually explains
this format. The only reference I can find to it is some sample go code.
The attached tcpdump.log contains a tcp dump between the mesos master and my
client collected using the following command {{tcpdump -xx -n -i lo "dst port
5050" or "src port 5050" 2>&1 | tee /tmp/tcpdump.log}}
was:
When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
some anomalous bytes contained in the chunked stream that appear to be causing
problems when I attempting to integrate.
Attached are two log files. app.log represents my application trying to connect
to mesos using RxNetty. Netty has been configured to log all data it
sends/receives over the wire this can be seen in the byte blocks in the log.
tcpdump.log contains a tcp dump between the mesos master and my client.
The client is constructing a protobuf in java for the subscribe call
{code:java}
final Call subscribeCall = Call.newBuilder()
.setType(Call.Type.SUBSCRIBE)
.setSubscribe(
Call.Subscribe.newBuilder()
.setFrameworkInfo(
Protos.FrameworkInfo.newBuilder()
.setUser("bill")
.setName("testing_this_shit_out")
.build()
)
)
.build();
{code}
lient sends the protobuf to mesos with the following request headers:
{code}
POST /api/v1/scheduler HTTP/1.1
Content-Type: application/x-protobuf
Accept: application/json
Content-Length: 35
Host: localhost:5050
User-Agent: RxNetty Client
{code}
The body is then serialized via protobuf and sent.
The response from the mesos master has the following headers:
{code}
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Wed, 30 Sep 2015 21:07:16 GMT
Content-Type: application/json
{code}
followed by
{code}
\r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028&
[jira] [Commented] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938932#comment-14938932
]
Ben Whitehead commented on MESOS-3562:
--
[~vinodkone] [~anandmazumdar] [~ijimenez] Ping
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to this framework generated by Mesos
> are streamed on this connection. Master encodes each Event in RecordIO
> format, i.e., string representation of length of the event followed by JSON
> or binary Protobuf (possibly compressed) encoded event.
> {quote}
> There is no specification I've been able to find online that actually
> explains this format. The only reference I can find to it is some sample go
> code.
> The attached tcpdump.log contains a tcp dump between the mesos master and my
> client collected using the following command {{tcpdump -xx -n -i lo "dst port
> 5050" or "src port 5050" 2>&1 | tee /tmp/tcpdump.log}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938944#comment-14938944
]
Benjamin Mahler commented on MESOS-3562:
This is just chunked encoding per the Transfer-Encoding header, your library
should de-chunk for you:
The first chunk has size 0x6c == 108 bytes of data.
The first chunk in this case contains a single record from our recordio format:
"104\n..." which is 104 + strlen("104\n") == 108.
We send you data in recordio format. That data may or may not be transfer
encode with chunked encoding. Ideally we set the Content-Type to express that
it is recordio wrapped, rather than just saying json, curious to see how other
streaming APIs set content type.
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to this framework generated by Mesos
> are streamed on this connection. Master encodes each Event in RecordIO
> format, i.e., string representation of length of the event followed by JSON
> or binary Protobuf (possibly compressed) encoded event.
> {quote}
> There is no specification I've been able to find online that actually
> explains this format. The only reference I can find to it is some sample go
> code.
> The attached tcpdump.log contains a tcp dump between the mesos master and my
> client collected using the following command {{tcpdump -xx -n -i lo "dst port
> 5050" or "src port 5050" 2>&1 | tee /tmp/tcpdump.log}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938944#comment-14938944
]
Benjamin Mahler edited comment on MESOS-3562 at 9/30/15 9:58 PM:
-
This is just chunked encoding per the Transfer-Encoding header, your library
should de-chunk for you:
The first chunk has size 0x6c == 108 bytes of data.
The first chunk in this case contains a single record of 104 bytes from our
recordio format: "104\n...". This chunk has length 104 + strlen("104\n") == 108.
We send you data in recordio format. That data may or may not be transfer
encode with chunked encoding. Ideally we set the Content-Type to express that
it is recordio wrapped, rather than just saying json, curious to see how other
streaming APIs set content type.
was (Author: bmahler):
This is just chunked encoding per the Transfer-Encoding header, your library
should de-chunk for you:
The first chunk has size 0x6c == 108 bytes of data.
The first chunk in this case contains a single record from our recordio format:
"104\n..." which is 104 + strlen("104\n") == 108.
We send you data in recordio format. That data may or may not be transfer
encode with chunked encoding. Ideally we set the Content-Type to express that
it is recordio wrapped, rather than just saying json, curious to see how other
streaming APIs set content type.
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to this framework generated by Mesos
> are streamed on this c
[jira] [Commented] (MESOS-3493) benchmark for declining offers
[ https://issues.apache.org/jira/browse/MESOS-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938950#comment-14938950 ] James Peach commented on MESOS-3493: I updated the benchmark to be less tacky and dropped https://reviews.apache.org/r/38648/ since that it no longer needed. > benchmark for declining offers > -- > > Key: MESOS-3493 > URL: https://issues.apache.org/jira/browse/MESOS-3493 > Project: Mesos > Issue Type: Improvement > Components: test >Reporter: James Peach >Priority: Minor > > I wrote a benchmark that can be used to demonstrate the performance issues > addressed in MESOS-3052, MESOS-3051, MESOS-3157 and MESOS-3075. The benchmark > simulates a number of frameworks that start declining all offers once they > reach the limit of work they need to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-3563) Revocable task CPU shows as zero in /state.json
Maxim Khutornenko created MESOS-3563:
Summary: Revocable task CPU shows as zero in /state.json
Key: MESOS-3563
URL: https://issues.apache.org/jira/browse/MESOS-3563
Project: Mesos
Issue Type: Bug
Reporter: Maxim Khutornenko
The slave's state.json reports revocable task resources as zero:
{noformat}
resources: {
cpus: 0,
disk: 3071,
mem: 1248,
ports: "[31715-31715]"
},
{noformat}
Also, there is no indication that a task uses revocable CPU. It would be great
to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3521) Support for hardware topology and resources description
[
https://issues.apache.org/jira/browse/MESOS-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938967#comment-14938967
]
Felix Abecassis commented on MESOS-3521:
[~qianzhang]: thank you, we were not aware of MESOS-3401, and indeed the
motivation seems to be the same. I tend to agree with everything you said in
your last comment on MESOS-3401.
This bug proposes a new CLI option to essentially provide attributes on
resources; we could also extend the existing attribute syntax, but it's
currently too limited.
I think there are 2 key points here:
- We probably want to leave resources unchanged when sending offers, and
frameworks can optionally use advanced information if they want to be smarter.
- We need a way to attach attributes to resources, there are multiple ways to
achieve this.
> Support for hardware topology and resources description
> ---
>
> Key: MESOS-3521
> URL: https://issues.apache.org/jira/browse/MESOS-3521
> Project: Mesos
> Issue Type: Improvement
>Reporter: Jonathan Calmels
>
> In heterogeneous clusters, tasks sometimes have strong constraints on the
> type of hardware they need to execute on. The current solution is to use
> custom attributes to describe resources on the agents.
> While this solution works, the current attribute format is somehow
> constraining and requires workarounds on both the agent and the scheduler
> (e.g. Base64 RFC 6920). In addition, we often need to encode the link between
> an attribute and a resource which is inherently error prone and
> implementation defined from one scheduler to another.
> I would like to propose a unified format to expose hardware topology and
> resources information at the agent level.
> This format would effectively extend the following specification:
> [http://mesos.apache.org/documentation/attributes-resources/]
> {code:title=specification}
> kv : text ":" ( range | kvSet | scalar | text )
> kvSet : "{" kv ( "," kv )* "}"
> info : ( range | text | "*" ) : kvSet
> infoSet : "{" info ( "," info )* "}"
> resourceName : text | "*"
> resourcesInfoValue : text "(" resourceName ")" ":" infoSet
> resourcesInfo : resourcesInfoValue ( ";" resourcesInfoValue )*
> {code}
> {code:javascript|title=example}
> --resources= gpus:{card0, card1};ports:[0-100];cpus:8
> --resources_info= nvidia(gpus): {
> card0: {
> uuid: GPU-34e8d7ba-0e4d-ac00-6852-695d5d404f51,
> name: GeForce_GTX_980,
> path: /dev/nvidia0,
> clocks: {
> graphic: 1392,
> sm: 1392
> }
> },
> card1: {
> uuid: GPU-12e457ba-0f4e-bf01-3452-674a5b212c21,
> name: GeForce_GTX_970,
> path: /dev/nvidia1
> clocks: {
> graphic: 1392,
> sm: 1392
> }
> },
> *: {
> driver: 352.39
> }
>};
>services(ports): {
> [0-79]: {
> type: daemon,
> user: root
> },
> [80-100]: {
> type: web_services,
> user: www-data
> }
>};
>procs(cpus): {
> *: {
> name: Intel_i7_6700K,
> frequency: 4,
> cache: 8
> }
>}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-3564) release candidate builds don't create Python cli libraries
David Robinson created MESOS-3564:
-
Summary: release candidate builds don't create Python cli libraries
Key: MESOS-3564
URL: https://issues.apache.org/jira/browse/MESOS-3564
Project: Mesos
Issue Type: Bug
Components: build
Reporter: David Robinson
Priority: Minor
Twitter creates internal release candidate builds by modifying the version
define in configure.ac:
{noformat}
sed -i~ "s/\[mesos\], \[.*\]/[mesos], [0.26.0-rc1]/" configure.ac
{noformat}
{noformat}
--- configure.ac~ 2015-09-30 11:22:30.0 -0700
+++ configure.ac2015-09-30 14:24:01.018998406 -0700
@@ -18,7 +18,7 @@
# Process this file with autoconf to produce a configure script.
AC_PREREQ([2.61])
-AC_INIT([mesos], [0.26.0])
+AC_INIT([mesos], [0.26.0-rc1])
# Have autoconf setup some variables related to the system.
AC_CANONICAL_HOST
{noformat}
The method described above has recently started causing problems for Python
wheels. Changing the version as per above causes wheel unpacking to fail:
{noformat}
18:15:20 DEBUG: cd python/dist && \
18:15:20 DEBUG: for whl in python/dist/mesos-0.26.0-rc1-py2-none-any.whl
python/dist/mesos.cli-0.26.0-rc1-py2-none-any.whl
python/dist/mesos.interface-0.26.0-rc1-py2-none-any.whl
python/dist/mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl; do
\
18:15:20 DEBUG:
PYTHONPATH=/var/tmp/mesos-0.26.0-rc1-root-mockbuild/usr/local/lib/python2.7/site-packages:/builddir/build/BUILD/mesos-0.26.0/3rdparty/distribute-0.6.26:/builddir/build/BUILD/mesos-0.26.0/3rdparty/pip-1.5.6:/builddir/build/BUILD/mesos-0.26.0/3rdparty/wheel-0.24.0
\
18:15:20 DEBUG:
PYTHONUSERBASE=/var/tmp/mesos-0.26.0-rc1-root-mockbuild/usr/local
\
18:15:20 DEBUG: /usr/bin/python2.7 -c "import pip; pip.main()" install
\
18:15:20 DEBUG: --user \
18:15:20 DEBUG:
--find-links=file:///builddir/build/BUILD/mesos-0.26.0/src/python/dist \
18:15:20 DEBUG: /builddir/build/BUILD/mesos-0.26.0/src/$whl;
\
18:15:20 DEBUG: done
18:15:21 DEBUG: Requirement
'/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos-0.26.0-rc1-py2-none-any.whl'
looks like a filename, but the file does not exist
18:15:21 DEBUG: mesos-0.26.0-rc1-py2-none-any.whl is not a supported wheel on
this platform.
18:15:21 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
18:15:21 DEBUG: Requirement
'/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.cli-0.26.0-rc1-py2-none-any.whl'
looks like a filename, but the file does not exist
18:15:21 DEBUG: mesos.cli-0.26.0-rc1-py2-none-any.whl is not a supported wheel
on this platform.
18:15:21 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
18:15:22 DEBUG: Requirement
'/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.interface-0.26.0-rc1-py2-none-any.whl'
looks like a filename, but the file does not exist
18:15:22 DEBUG: mesos.interface-0.26.0-rc1-py2-none-any.whl is not a supported
wheel on this platform.
18:15:22 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
18:15:22 DEBUG: Requirement
'/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl'
looks like a filename, but the file does not exist
18:15:22 DEBUG: mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl is not a
supported wheel on this platform.
{noformat}
Something is sanitizing the version, so 0.26.0-rc1 becomes 0.26.0_rc1:
{noformat}[builder@ci_server ~]# ls -la
/var/lib/mock/mesos-5-x86_64/root/builddir/build/BUILD/mesos-0.26.0/src/python/dist/
total 181772
-rw-r--r--. 1 builder builder 1706 Sep 30 18:15 mesos-0.26.0_rc1-py2.7.egg
-rw-r--r--. 1 builder builder 2171 Sep 30 18:15
mesos-0.26.0_rc1-py2-none-any.whl
-rw-r--r--. 1 builder builder 9241 Sep 30 18:15
mesos.cli-0.26.0_rc1-py2.7.egg
-rw-r--r--. 1 builder builder 6440 Sep 30 18:15
mesos.cli-0.26.0_rc1-py2-none-any.whl
-rw-r--r--. 1 builder builder50481 Sep 30 18:15
mesos.interface-0.26.0_rc1-py2.7.egg
-rw-r--r--. 1 builder builder25667 Sep 30 18:15
mesos.interface-0.26.0_rc1-py2-none-any.whl
-rw-r--r--. 1 builder builder 93007783 Sep 30 18:16
mesos.native-0.26.0_rc1-cp27-none-linux_x86_64.whl
-rw-r--r--. 1 builder builder 93008349 Sep 30 18:15
mesos.native-0.26.0_rc1-py2.7-linux-x86_64.egg
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3564) release candidate builds fail to unpack Python wheels
[
https://issues.apache.org/jira/browse/MESOS-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Robinson updated MESOS-3564:
--
Summary: release candidate builds fail to unpack Python wheels (was:
release candidate builds don't create Python cli libraries)
> release candidate builds fail to unpack Python wheels
> -
>
> Key: MESOS-3564
> URL: https://issues.apache.org/jira/browse/MESOS-3564
> Project: Mesos
> Issue Type: Bug
> Components: build
>Reporter: David Robinson
>Priority: Minor
>
> Twitter creates internal release candidate builds by modifying the version
> define in configure.ac:
> {noformat}
> sed -i~ "s/\[mesos\], \[.*\]/[mesos], [0.26.0-rc1]/" configure.ac
> {noformat}
> {noformat}
> --- configure.ac~ 2015-09-30 11:22:30.0 -0700
> +++ configure.ac 2015-09-30 14:24:01.018998406 -0700
> @@ -18,7 +18,7 @@
> # Process this file with autoconf to produce a configure script.
>
> AC_PREREQ([2.61])
> -AC_INIT([mesos], [0.26.0])
> +AC_INIT([mesos], [0.26.0-rc1])
>
> # Have autoconf setup some variables related to the system.
> AC_CANONICAL_HOST
> {noformat}
> The method described above has recently started causing problems for Python
> wheels. Changing the version as per above causes wheel unpacking to fail:
> {noformat}
> 18:15:20 DEBUG: cd python/dist && \
> 18:15:20 DEBUG: for whl in python/dist/mesos-0.26.0-rc1-py2-none-any.whl
> python/dist/mesos.cli-0.26.0-rc1-py2-none-any.whl
> python/dist/mesos.interface-0.26.0-rc1-py2-none-any.whl
> python/dist/mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl; do
>\
> 18:15:20 DEBUG:
> PYTHONPATH=/var/tmp/mesos-0.26.0-rc1-root-mockbuild/usr/local/lib/python2.7/site-packages:/builddir/build/BUILD/mesos-0.26.0/3rdparty/distribute-0.6.26:/builddir/build/BUILD/mesos-0.26.0/3rdparty/pip-1.5.6:/builddir/build/BUILD/mesos-0.26.0/3rdparty/wheel-0.24.0
> \
> 18:15:20 DEBUG:
> PYTHONUSERBASE=/var/tmp/mesos-0.26.0-rc1-root-mockbuild/usr/local
> \
> 18:15:20 DEBUG: /usr/bin/python2.7 -c "import pip; pip.main()"
> install \
> 18:15:20 DEBUG: --user \
> 18:15:20 DEBUG:
> --find-links=file:///builddir/build/BUILD/mesos-0.26.0/src/python/dist \
> 18:15:20 DEBUG: /builddir/build/BUILD/mesos-0.26.0/src/$whl;
> \
> 18:15:20 DEBUG: done
> 18:15:21 DEBUG: Requirement
> '/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos-0.26.0-rc1-py2-none-any.whl'
> looks like a filename, but the file does not exist
> 18:15:21 DEBUG: mesos-0.26.0-rc1-py2-none-any.whl is not a supported wheel on
> this platform.
> 18:15:21 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
> 18:15:21 DEBUG: Requirement
> '/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.cli-0.26.0-rc1-py2-none-any.whl'
> looks like a filename, but the file does not exist
> 18:15:21 DEBUG: mesos.cli-0.26.0-rc1-py2-none-any.whl is not a supported
> wheel on this platform.
> 18:15:21 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
> 18:15:22 DEBUG: Requirement
> '/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.interface-0.26.0-rc1-py2-none-any.whl'
> looks like a filename, but the file does not exist
> 18:15:22 DEBUG: mesos.interface-0.26.0-rc1-py2-none-any.whl is not a
> supported wheel on this platform.
> 18:15:22 DEBUG: Storing debug log for failure in /builddir/.pip/pip.log
> 18:15:22 DEBUG: Requirement
> '/builddir/build/BUILD/mesos-0.26.0/src/python/dist/mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl'
> looks like a filename, but the file does not exist
> 18:15:22 DEBUG: mesos.native-0.26.0-rc1-cp27-none-linux_x86_64.whl is not a
> supported wheel on this platform.
> {noformat}
> Something is sanitizing the version, so 0.26.0-rc1 becomes 0.26.0_rc1:
> {noformat}[builder@ci_server ~]# ls -la
> /var/lib/mock/mesos-5-x86_64/root/builddir/build/BUILD/mesos-0.26.0/src/python/dist/
> total 181772
> -rw-r--r--. 1 builder builder 1706 Sep 30 18:15 mesos-0.26.0_rc1-py2.7.egg
> -rw-r--r--. 1 builder builder 2171 Sep 30 18:15
> mesos-0.26.0_rc1-py2-none-any.whl
> -rw-r--r--. 1 builder builder 9241 Sep 30 18:15
> mesos.cli-0.26.0_rc1-py2.7.egg
> -rw-r--r--. 1 builder builder 6440 Sep 30 18:15
> mesos.cli-0.26.0_rc1-py2-none-any.whl
> -rw-r--r--. 1 builder builder50481 Sep 30 18:15
> mesos.interface-0.26.0_rc1-py2.7.egg
> -rw-r--r--. 1 builder builder25667 Sep 30 18:15
> mesos.interface-0.26.0_rc1-py2-none-any.whl
> -rw-r--r--. 1 builder builder 93007783 Sep 30 18:16
> mesos.native-0.26.0_rc1-cp27-none-linux_x86_64.whl
> -rw-r--r--. 1 builder builder 93008349 Sep 30 18:15
> mesos.native-0.26.0_rc1-py2.7-linux-x86_
[jira] [Created] (MESOS-3565) Show framework role in the mesos master webui
Vinod Kone created MESOS-3565: - Summary: Show framework role in the mesos master webui Key: MESOS-3565 URL: https://issues.apache.org/jira/browse/MESOS-3565 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Currently the "frameworks" page in the webui shows id, host, and username from FrameworkInfo. It would be really useful to also display "role" info when diagnosing multi-framework allocation issues. Even better would be a way to show all the fields of FrameworkInfo (since we keep adding new fields) but we can track that in a separate ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939021#comment-14939021
]
Niklas Quarfot Nielsen commented on MESOS-3563:
---
Can you share the full state.json?
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kone updated MESOS-3563:
--
Assignee: Vinod Kone
Sprint: Twitter Mesos Q3 Sprint 6
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939030#comment-14939030
]
Maxim Khutornenko commented on MESOS-3563:
--
It's just too big to sanitize :) Is there anything in particular you are
interested in?
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939044#comment-14939044
]
Niklas Quarfot Nielsen commented on MESOS-3563:
---
I didn't understand the full context of your problem; assume you are using the
fixed resource estimator? At any regard, looks like Vinod is on it.
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939046#comment-14939046
]
Maxim Khutornenko commented on MESOS-3563:
--
Correct, fixed resource estimator.
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3479) COMMAND Health Checks are not executed if the timeout is exceeded
[ https://issues.apache.org/jira/browse/MESOS-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939054#comment-14939054 ] Gabriel Hartmann commented on MESOS-3479: - [~haosd...@gmail.com]: I'm seeing this issue as well. The config is like this: "gracePeriodSeconds": 300, "intervalSeconds": 60, "timeoutSeconds": 60, "maxConsecutiveFailures": 3 We fail early on 3 times in a row. Then the 4th attempt takes more than 60s to eventually fail. While it's running a 5th attempt is started (it succeeds). All this occurs before expiration of the grace period. The 5th attempt is the last attempt. No more health checks are made. Marathon never receives a health check report. Is there an ETA for a fix for this? It's very disruptive to frameworks converging to a healthy or unhealthy state. Marathon in this case will see the framework as having 1 running task, with 0 staging, 0 healthy, and 0 unhealthy. > COMMAND Health Checks are not executed if the timeout is exceeded > - > > Key: MESOS-3479 > URL: https://issues.apache.org/jira/browse/MESOS-3479 > Project: Mesos > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Matthias Veit >Assignee: haosdent >Priority: Critical > > The issue first appeared as Marathon Bug: See here for reference: > https://github.com/mesosphere/marathon/issues/2179. > A COMMAND health check is defined with a timeout of 20 seconds. > The command itself takes longer than 20 seconds to execute. > Current behavior: > - The mesos health check process get's killed, but the defined command > process not (in the example the curl command returns after 21 seconds). > - The check attempt is considered healthy, if the timeout is exceeded > - The health check stops and is not executed any longer > Expected behavior: > - The defined health check command is killed, when the timeout is exceeded > - The check attempt is considered Unhealthy, if the timeout is exceeded > - The health check does not stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3479) COMMAND Health Checks are not executed if the timeout is exceeded
[ https://issues.apache.org/jira/browse/MESOS-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939054#comment-14939054 ] Gabriel Hartmann edited comment on MESOS-3479 at 9/30/15 11:29 PM: --- [~haosd...@gmail.com]: I'm seeing this issue as well. The config is like this: "gracePeriodSeconds": 300, "intervalSeconds": 60, "timeoutSeconds": 60, "maxConsecutiveFailures": 3 We fail early on 3 times in a row. Then the 4th attempt takes more than 60s to eventually fail/timeout. While it's running a 5th attempt is started (it succeeds). All this occurs before expiration of the grace period. The 5th attempt is the last attempt. No more health checks are made. Marathon never receives a health check report. Is there an ETA for a fix for this? It's very disruptive to have frameworks which never converge to a healthy or unhealthy state. Marathon in this case will see the framework as having 1 running task, with 0 staging, 0 healthy, and 0 unhealthy. was (Author: gabriel.hartm...@gmail.com): [~haosd...@gmail.com]: I'm seeing this issue as well. The config is like this: "gracePeriodSeconds": 300, "intervalSeconds": 60, "timeoutSeconds": 60, "maxConsecutiveFailures": 3 We fail early on 3 times in a row. Then the 4th attempt takes more than 60s to eventually fail/timeout. While it's running a 5th attempt is started (it succeeds). All this occurs before expiration of the grace period. The 5th attempt is the last attempt. No more health checks are made. Marathon never receives a health check report. Is there an ETA for a fix for this? It's very disruptive to frameworks converging to a healthy or unhealthy state. Marathon in this case will see the framework as having 1 running task, with 0 staging, 0 healthy, and 0 unhealthy. > COMMAND Health Checks are not executed if the timeout is exceeded > - > > Key: MESOS-3479 > URL: https://issues.apache.org/jira/browse/MESOS-3479 > Project: Mesos > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Matthias Veit >Assignee: haosdent >Priority: Critical > > The issue first appeared as Marathon Bug: See here for reference: > https://github.com/mesosphere/marathon/issues/2179. > A COMMAND health check is defined with a timeout of 20 seconds. > The command itself takes longer than 20 seconds to execute. > Current behavior: > - The mesos health check process get's killed, but the defined command > process not (in the example the curl command returns after 21 seconds). > - The check attempt is considered healthy, if the timeout is exceeded > - The health check stops and is not executed any longer > Expected behavior: > - The defined health check command is killed, when the timeout is exceeded > - The check attempt is considered Unhealthy, if the timeout is exceeded > - The health check does not stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-3479) COMMAND Health Checks are not executed if the timeout is exceeded
[ https://issues.apache.org/jira/browse/MESOS-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939054#comment-14939054 ] Gabriel Hartmann edited comment on MESOS-3479 at 9/30/15 11:29 PM: --- [~haosd...@gmail.com]: I'm seeing this issue as well. The config is like this: "gracePeriodSeconds": 300, "intervalSeconds": 60, "timeoutSeconds": 60, "maxConsecutiveFailures": 3 We fail early on 3 times in a row. Then the 4th attempt takes more than 60s to eventually fail/timeout. While it's running a 5th attempt is started (it succeeds). All this occurs before expiration of the grace period. The 5th attempt is the last attempt. No more health checks are made. Marathon never receives a health check report. Is there an ETA for a fix for this? It's very disruptive to frameworks converging to a healthy or unhealthy state. Marathon in this case will see the framework as having 1 running task, with 0 staging, 0 healthy, and 0 unhealthy. was (Author: gabriel.hartm...@gmail.com): [~haosd...@gmail.com]: I'm seeing this issue as well. The config is like this: "gracePeriodSeconds": 300, "intervalSeconds": 60, "timeoutSeconds": 60, "maxConsecutiveFailures": 3 We fail early on 3 times in a row. Then the 4th attempt takes more than 60s to eventually fail. While it's running a 5th attempt is started (it succeeds). All this occurs before expiration of the grace period. The 5th attempt is the last attempt. No more health checks are made. Marathon never receives a health check report. Is there an ETA for a fix for this? It's very disruptive to frameworks converging to a healthy or unhealthy state. Marathon in this case will see the framework as having 1 running task, with 0 staging, 0 healthy, and 0 unhealthy. > COMMAND Health Checks are not executed if the timeout is exceeded > - > > Key: MESOS-3479 > URL: https://issues.apache.org/jira/browse/MESOS-3479 > Project: Mesos > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Matthias Veit >Assignee: haosdent >Priority: Critical > > The issue first appeared as Marathon Bug: See here for reference: > https://github.com/mesosphere/marathon/issues/2179. > A COMMAND health check is defined with a timeout of 20 seconds. > The command itself takes longer than 20 seconds to execute. > Current behavior: > - The mesos health check process get's killed, but the defined command > process not (in the example the curl command returns after 21 seconds). > - The check attempt is considered healthy, if the timeout is exceeded > - The health check stops and is not executed any longer > Expected behavior: > - The defined health check command is killed, when the timeout is exceeded > - The check attempt is considered Unhealthy, if the timeout is exceeded > - The health check does not stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939069#comment-14939069
]
Vinod Kone commented on MESOS-3563:
---
Looks like this is done intentionally, to preserve backwards compatibility.
https://github.com/apache/mesos/blob/master/src/common/http.cpp#L73
{quote}
// To maintain backwards compatibility we exclude revocable
// resources in the reporting.
{quote}
This was done as part of fixing the resource representation in MESOS-2838. cc
[~xujyan]
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2613) Change docker rm command
[ https://issues.apache.org/jira/browse/MESOS-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939079#comment-14939079 ] Greg Mann commented on MESOS-2613: -- Review here: https://reviews.apache.org/r/38910/ > Change docker rm command > > > Key: MESOS-2613 > URL: https://issues.apache.org/jira/browse/MESOS-2613 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Reporter: Mike Michel >Assignee: Greg Mann >Priority: Minor > Labels: mesosphere > > Right now it seems Mesos is using „docker rm –f ID“ to delete containers so > bind mounts are not deleted. This means thousands of dirs in > /var/lib/docker/vfs/dir I would like to have the option to change it to > „docker rm –f –v ID“ This deletes bind mounts but not persistant volumes. > Best, > Mike -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-3563) Revocable task CPU shows as zero in /state.json
[
https://issues.apache.org/jira/browse/MESOS-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939096#comment-14939096
]
Vinod Kone commented on MESOS-3563:
---
So I think we could/should add the revocable resources information in the
endpoints that display "Resources".
Couple of options
#1: have a different "revocable_resources" object in addition to the
"resources" object in the JSON.
{code}
revocable_resources: {
cpus: 2,
...
}
{code}
#2: keep one "resources" object but add new keys "cpus_revocable"
{code}
resources : {
cpus: 2,
cpus_revocable: 2,
...
}
{code}
Any preferences? #1 seems easier to implement and maintain. #2 seems closer to
what /metrics/snapshot has (slave/cpus_total, slave/cpus_revocable_total)
> Revocable task CPU shows as zero in /state.json
> ---
>
> Key: MESOS-3563
> URL: https://issues.apache.org/jira/browse/MESOS-3563
> Project: Mesos
> Issue Type: Bug
>Reporter: Maxim Khutornenko
>Assignee: Vinod Kone
>
> The slave's state.json reports revocable task resources as zero:
> {noformat}
> resources: {
> cpus: 0,
> disk: 3071,
> mem: 1248,
> ports: "[31715-31715]"
> },
> {noformat}
> Also, there is no indication that a task uses revocable CPU. It would be
> great to have this type of info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939173#comment-14939173
]
Ben Whitehead commented on MESOS-3562:
--
This isn't really standard chunks though, there are chunks within chunks and
the configuration of the client would have to know that.
What is the motivation behind using recordio format rather than standard chunk
encoding defined in [RFC 2616 Sec.
3|http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html]? If standard encoding
were used then every HTTP client would already have the necessary understanding
to know how to deal with the chunks.
Where is the specification for what recordio format is? I have not been able to
find anything online.
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stream from the new http api see
> [this|http://mail-archives.apache.org/mod_mbox/mesos-user/201508.mbox/%3c94d2c9e8-2fe8-4c11-b0d3-859dac654...@me.com%3E]
> message from the mailing list.
> In the [Design
> Doc|https://docs.google.com/document/d/1pnIY_HckimKNvpqhKRhbc9eSItWNFT-priXh_urR-T0/edit#]
> there is a statement that said
> {quote}
> All subsequent events that are relevant to this framework generated by Mesos
> are streamed on this connection. Master encodes each Event in RecordIO
> format, i.e., string representation of length of the event followed by JSON
> or binary Protobuf (possibly compressed) encoded event.
> {quote}
> There is no specification I've been able to find online that actually
> explains this format. The only reference I can find to it is some sample go
> code.
> The attached tcpdump.log contains a tcp dump between the mesos master and my
> client collected using the following command {{tcpdump -xx -n -i lo "dst port
> 5050" or "src port 5050" 2>&1 | tee /tmp/tcpdump.log}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3562) Anomalous bytes in stream from HTTPI Api
[
https://issues.apache.org/jira/browse/MESOS-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939194#comment-14939194
]
Anand Mazumdar commented on MESOS-3562:
---
[~BenWhitehead] There seems to be some confusion here. Comments Inline.
> This isn't really standard chunks though, there are chunks within chunks and
> the configuration of the client would have to know that.
Can you elaborate a bit more on what do you mean by chunks between chunks here
? We strictly adhere to the standard chunk encoding format defined in RFC 2616.
The only difference here is that the {{data}} in chunks itself is encoded in
{{RecordIO}} format.
> What is the motivation behind using recordio format ?
We wanted a way to delimit two events for JSON/Protobuf responses and RecordIO
format allowed us to do that. We could have gone away with RecordIO for JSON
though by just delimiting on {{\n}} but that would have made it inconsistent in
behavior when compared to Protobuf Responses.
> If standard encoding were used then every HTTP client would already have the
> necessary understanding to know how to deal with the chunks.
We use standard chunk encoding as defined in RFC. What do you mean here ?
> Where is the specification for what recordio format is? I have not been able
> to find anything online.
We should add more information on this in our docs. For now, till we do that,
here is a brief description on what the format looks like:
{code}
5\n
hello
6\n
world!
{code}
Ideally, whatever client you are using should do the de-chunking for you. You
should get this back from the client i.e. just the {{RecordIO}} encoded data.
{code}
104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
{code}
cc'ing [~bmahler] If I missed anything.
> Anomalous bytes in stream from HTTPI Api
>
>
> Key: MESOS-3562
> URL: https://issues.apache.org/jira/browse/MESOS-3562
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API
>Affects Versions: 0.24.0
> Environment: Linux 3.16.7-24-desktop #1 SMP PREEMPT Mon Aug 3
> 14:37:06 UTC 2015 (ec183cc) x86_64 x86_64 GNU/Linux
> Mesos 0.24.0
> gcc (SUSE Linux) 4.8.3 20140627 [gcc-4_8-branch revision 212064]
>Reporter: Ben Whitehead
>Priority: Blocker
> Labels: http, wireprotocol
> Attachments: app.log, tcpdump.log
>
>
> When connecting to the new HTTP Api and attempting to {{SUBSCRIBE}} there are
> some anomalous bytes contained in the chunked stream that appear to be
> causing problems when I attempting to integrate.
> Attached are two log files. app.log represents my application trying to
> connect to mesos using RxNetty. Netty has been configured to log all data it
> sends/receives over the wire this can be seen in the byte blocks in the log.
> The client is constructing a protobuf in java for the subscribe call
> {code:java}
> final Call subscribeCall = Call.newBuilder()
> .setType(Call.Type.SUBSCRIBE)
> .setSubscribe(
> Call.Subscribe.newBuilder()
> .setFrameworkInfo(
> Protos.FrameworkInfo.newBuilder()
> .setUser("bill")
> .setName("testing_this_shit_out")
> .build()
> )
> )
> .build();
> {code}
>
> lient sends the protobuf to mesos with the following request headers:
> {code}
> POST /api/v1/scheduler HTTP/1.1
> Content-Type: application/x-protobuf
> Accept: application/json
> Content-Length: 35
> Host: localhost:5050
> User-Agent: RxNetty Client
> {code}
> The body is then serialized via protobuf and sent.
> The response from the mesos master has the following headers:
> {code}
> HTTP/1.1 200 OK
> Transfer-Encoding: chunked
> Date: Wed, 30 Sep 2015 21:07:16 GMT
> Content-Type: application/json
> {code}
> followed by
> {code}
> \r\n\r\n6c\r\n104\n{"subscribed":{"framework_id":{"value":"20150930-103028-16777343-5050-11742-0028"}},"type":"SUBSCRIBED"}
> {code}
> The {{\r\n\r\n}} is expected for standard http bodies, how ever {{6c\r\n}}
> doesn't appear to be attached to anything. {{104}} is the correct length of
> the Subscribe events JSON.
> What is this extra number and why is it there?
> This is not the first time confusion has come up related to the wire format
> for the event stre
[jira] [Created] (MESOS-3566) Add a section to the Scheduler HTTP API docs around RecordIO specification
Anand Mazumdar created MESOS-3566:
-
Summary: Add a section to the Scheduler HTTP API docs around
RecordIO specification
Key: MESOS-3566
URL: https://issues.apache.org/jira/browse/MESOS-3566
Project: Mesos
Issue Type: Improvement
Reporter: Anand Mazumdar
Since the {{RecordIO}} format is not that widely used, searching for it online
does not offer much help.
- It would be good if we can add to the docs, a small section on its
specification for framework developers.
- Bonus points, if we can have a simple code snippet in C++/Java on reading a
{{RecordIO}} response to help developers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3479) COMMAND Health Checks are not executed if the timeout is exceeded
[ https://issues.apache.org/jira/browse/MESOS-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939396#comment-14939396 ] haosdent commented on MESOS-3479: - Got it. I would try to submit a patch for this at this weekend. > COMMAND Health Checks are not executed if the timeout is exceeded > - > > Key: MESOS-3479 > URL: https://issues.apache.org/jira/browse/MESOS-3479 > Project: Mesos > Issue Type: Bug >Affects Versions: 0.23.0 >Reporter: Matthias Veit >Assignee: haosdent >Priority: Critical > > The issue first appeared as Marathon Bug: See here for reference: > https://github.com/mesosphere/marathon/issues/2179. > A COMMAND health check is defined with a timeout of 20 seconds. > The command itself takes longer than 20 seconds to execute. > Current behavior: > - The mesos health check process get's killed, but the defined command > process not (in the example the curl command returns after 21 seconds). > - The check attempt is considered healthy, if the timeout is exceeded > - The health check stops and is not executed any longer > Expected behavior: > - The defined health check command is killed, when the timeout is exceeded > - The check attempt is considered Unhealthy, if the timeout is exceeded > - The health check does not stop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-3565) Show framework role in the mesos master webui
[ https://issues.apache.org/jira/browse/MESOS-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] haosdent reassigned MESOS-3565: --- Assignee: haosdent > Show framework role in the mesos master webui > - > > Key: MESOS-3565 > URL: https://issues.apache.org/jira/browse/MESOS-3565 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Assignee: haosdent > > Currently the "frameworks" page in the webui shows id, host, and username > from FrameworkInfo. It would be really useful to also display "role" info > when diagnosing multi-framework allocation issues. > Even better would be a way to show all the fields of FrameworkInfo (since we > keep adding new fields) but we can track that in a separate ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2533) Support HTTP checks in Mesos health check program
[ https://issues.apache.org/jira/browse/MESOS-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939398#comment-14939398 ] haosdent commented on MESOS-2533: - Yes, I need rebase it first. > Support HTTP checks in Mesos health check program > - > > Key: MESOS-2533 > URL: https://issues.apache.org/jira/browse/MESOS-2533 > Project: Mesos > Issue Type: Bug >Reporter: Niklas Quarfot Nielsen >Assignee: haosdent > > Currently, only commands are supported but our health check protobuf enables > users to encode HTTP checks as well. We should wire up this in the health > check program or remove the http field from the protobuf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
