[jira] [Created] (MESOS-4587) Docker environment variables must be able to contain the equal sign
Martin Tapp created MESOS-4587:
--
Summary: Docker environment variables must be able to contain the
equal sign
Key: MESOS-4587
URL: https://issues.apache.org/jira/browse/MESOS-4587
Project: Mesos
Issue Type: Bug
Components: docker
Affects Versions: 0.25.0
Reporter: Martin Tapp
Priority: Minor
Note: Affects 0.26 and 0.27.
The Jupyter Docker all-spark-notebook uses equal sign ('=') in Docker ENV
declarations (for instance,
https://github.com/jupyter/docker-stacks/blob/master/all-spark-notebook/Dockerfile#L51).
This causes a mesos Unexpected Env format for 'ContainerConfig.Env' error.
The problem is the tokenization code at
https://github.com/apache/mesos/blob/21e080c5ae6ef03556c7a2b588e034a916c7a05a/src/docker/docker.cpp#L386
which needs to only look at the first equal sign. Docker ENV declarations can
also be empty.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-7292) Introduce a "sensitive mode" in Mesos which prevents leaks of sensitive data.
[
https://issues.apache.org/jira/browse/MESOS-7292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074870#comment-16074870
]
Martin Tapp commented on MESOS-7292:
Please make the POSSIBLY-SENSITIVE-DATA optional as noted in the code as we
can't identify anonymous tasks using ENV vars
(https://github.com/apache/mesos/blob/master/src/launcher/executor.cpp#L470).
Thanks
> Introduce a "sensitive mode" in Mesos which prevents leaks of sensitive data.
> -
>
> Key: MESOS-7292
> URL: https://issues.apache.org/jira/browse/MESOS-7292
> Project: Mesos
> Issue Type: Improvement
> Components: security
>Reporter: Alexander Rukletsov
> Labels: mesosphere, security
>
> Consider a following scenario. A user passes some sensitive data in an
> environment variable to a task. These data may be logged by Mesos components,
> e.g., executor as part of {{mesos-containerizer}} invocation. While this is
> useful for debugging, this might be an issue in some production environments.
> One of the solution is to have global "sensitive mode", that turns off
> logging of such sensitive data.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167933#comment-16167933 ] Martin Tapp commented on MESOS-7518: I've isolated that a new pip package (installed to both pip2 and pip3) causes the problem. Any idea why a mesos-slave or mesos-master would become invalidated, even having their apt packages uninstalled by Ubuntu 16.04? This never happens on nodes that don't have pip2/pip3 packages installed. > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253569#comment-16253569 ] Martin Tapp commented on MESOS-7518: Any comments on this, this has been a problem more than 6 months. Thanks > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
Martin Tapp created MESOS-7518: -- Summary: Mesos master and slave get uninstalled by ubuntu 16.04 Key: MESOS-7518 URL: https://issues.apache.org/jira/browse/MESOS-7518 Project: Mesos Issue Type: Bug Components: agent, master Affects Versions: 1.2.0 Environment: Ubuntu 16.04 Reporter: Martin Tapp Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and agent/slave sometimes get uninstalled automatically. This always happens after a reboot, but maybe every other day otherwise. We're running on bare metal servers and VMs with the same result. sudo service mesos-master status yields Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' to reload units. Same for mesos-slave. Any idea? Thanks -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Tapp updated MESOS-7518: --- Description: Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and agent/slave sometimes get uninstalled automatically. This always happens after a reboot, but maybe every other day otherwise. We're running on bare metal servers and VMs with the same result. sudo service mesos-master status yields Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' to reload units. Same for mesos-slave. Solution is to re-run our mesos provisioning automatically. apt-get install mesos re-installs it all the time. We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' with key 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' Any idea? Thanks was: Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and agent/slave sometimes get uninstalled automatically. This always happens after a reboot, but maybe every other day otherwise. We're running on bare metal servers and VMs with the same result. sudo service mesos-master status yields Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' to reload units. Same for mesos-slave. Any idea? Thanks > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, but maybe every other day otherwise. We're running on bare > metal servers and VMs with the same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Tapp updated MESOS-7518: --- Description: Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and agent/slave sometimes get uninstalled automatically. This always happens after a reboot, when you restart the service, and maybe every other day otherwise on it's own. We're running on bare metal servers and VMs with the same result. sudo service mesos-master status yields Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' to reload units. Same for mesos-slave. Solution is to re-run our mesos provisioning automatically. apt-get install mesos re-installs it all the time. We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' with key 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' Any idea? Thanks was: Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and agent/slave sometimes get uninstalled automatically. This always happens after a reboot, but maybe every other day otherwise. We're running on bare metal servers and VMs with the same result. sudo service mesos-master status yields Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' to reload units. Same for mesos-slave. Solution is to re-run our mesos provisioning automatically. apt-get install mesos re-installs it all the time. We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' with key 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' Any idea? Thanks > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374819#comment-16374819 ] Martin Tapp commented on MESOS-7518: Ok, so we're still experiencing this issue after reporting it 6 months ago, any help please? > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp >Priority: Major > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374819#comment-16374819 ] Martin Tapp edited comment on MESOS-7518 at 2/23/18 6:46 PM: - Ok, so we're still experiencing this issue after reporting it 10 months ago, any help please? was (Author: doctapp): Ok, so we're still experiencing this issue after reporting it 6 months ago, any help please? > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp >Priority: Major > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-7518) Mesos master and slave get uninstalled by ubuntu 16.04
[ https://issues.apache.org/jira/browse/MESOS-7518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439499#comment-16439499 ] Martin Tapp commented on MESOS-7518: Bump: seriously, any thoughts? Same problems with Mesos 1.4.1 > Mesos master and slave get uninstalled by ubuntu 16.04 > -- > > Key: MESOS-7518 > URL: https://issues.apache.org/jira/browse/MESOS-7518 > Project: Mesos > Issue Type: Bug > Components: agent, master >Affects Versions: 1.2.0 > Environment: Ubuntu 16.04 >Reporter: Martin Tapp >Priority: Major > > Since we've upgraded to Mesos 1.2 (from 1.0) on Ubuntu 16.04, the master and > agent/slave sometimes get uninstalled automatically. This always happens > after a reboot, when you restart the service, and maybe every other day > otherwise on it's own. We're running on bare metal servers and VMs with the > same result. > sudo service mesos-master status yields > Warning: mesos-master.service changed on disk. Run 'systemctl daemon-reload' > to reload units. > Same for mesos-slave. > Solution is to re-run our mesos provisioning automatically. apt-get install > mesos re-installs it all the time. > We're using apt source 'deb http://repos.mesosphere.io/ubuntu xenial main' > with key > 'http://keyserver.ubuntu.com/pks/lookup?op=get&fingerprint=on&search=0xE56151BF' > Any idea? > Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-1837) failed to determine cgroup for the 'cpu' subsystem
[ https://issues.apache.org/jira/browse/MESOS-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246393#comment-15246393 ] Martin Tapp commented on MESOS-1837: I have the same problem using Mesos 0.28.1 + Marathon 1.1.1. Using 3 mesos masters + 3 marathon masters with quorum set to 2. I0418 15:35:44.278214 4849 slave.cpp:3002] Handling status update TASK_FAILED (UUID: 44321081-f2f8-4f32-ac17-ea32d2925d62) for task dst-app.971f328f-059c-11e6-b40f-44a84205ccb0 of framework 4c095edc-a5c1-4540-9a07-72e4e3ec8c48- from executor(1)@10.48.176.41:45067 E0418 15:35:44.285468 4774 slave.cpp:3252] Failed to update resources for container 1954b97d-ee79-470d-885f-a4b13deae876 of executor 'dst-app.971f328f-059c-11e6-b40f-44a84205ccb0' running task dst-app.971f328f-059c-11e6-b40f-44a84205ccb0 on status update for terminal task, destroying container: Failed to determine cgroup for the 'cpu' subsystem: Failed to read /proc/35264/cgroup: Failed to open file '/proc/35264/cgroup': No such file or directory I0418 15:35:44.285833 4800 docker.cpp:1696] Destroying container '1954b97d-ee79-470d-885f-a4b13deae876' Any idea? Thanks > failed to determine cgroup for the 'cpu' subsystem > -- > > Key: MESOS-1837 > URL: https://issues.apache.org/jira/browse/MESOS-1837 > Project: Mesos > Issue Type: Bug > Components: docker >Affects Versions: 0.20.1 > Environment: Ubuntu 14.04 >Reporter: Chris Fortier >Assignee: Timothy Chen > > Attempting to launch Docker container with Marathon. Container is launched > then fails. > A search of /var/log/syslog reveals: > Sep 27 03:01:43 vagrant-ubuntu-trusty-64 mesos-slave[1409]: E0927 > 03:01:43.546957 1463 slave.cpp:2205] Failed to update resources for > container 8c2429d9-f090-4443-8108-0206ca37f3fd of executor > hello-world.970dbe74-45f2-11e4-8b1d-56847afe9799 running task > hello-world.970dbe74-45f2-11e4-8b1d-56847afe9799 on status update for > terminal task, destroying container: Failed to determine cgroup for the 'cpu' > subsystem: Failed to read /proc/9792/cgroup: Failed to open file > '/proc/9792/cgroup': No such file or directory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1837) failed to determine cgroup for the 'cpu' subsystem
[ https://issues.apache.org/jira/browse/MESOS-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247665#comment-15247665 ] Martin Tapp commented on MESOS-1837: [~haosdent] Thanks for the info. I noticed this is randomly appearing in logs when a task gets killed. > failed to determine cgroup for the 'cpu' subsystem > -- > > Key: MESOS-1837 > URL: https://issues.apache.org/jira/browse/MESOS-1837 > Project: Mesos > Issue Type: Bug > Components: docker >Affects Versions: 0.20.1 > Environment: Ubuntu 14.04 >Reporter: Chris Fortier >Assignee: Timothy Chen > > Attempting to launch Docker container with Marathon. Container is launched > then fails. > A search of /var/log/syslog reveals: > Sep 27 03:01:43 vagrant-ubuntu-trusty-64 mesos-slave[1409]: E0927 > 03:01:43.546957 1463 slave.cpp:2205] Failed to update resources for > container 8c2429d9-f090-4443-8108-0206ca37f3fd of executor > hello-world.970dbe74-45f2-11e4-8b1d-56847afe9799 running task > hello-world.970dbe74-45f2-11e4-8b1d-56847afe9799 on status update for > terminal task, destroying container: Failed to determine cgroup for the 'cpu' > subsystem: Failed to read /proc/9792/cgroup: Failed to open file > '/proc/9792/cgroup': No such file or directory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2374) Support relative host paths for container volumes
[ https://issues.apache.org/jira/browse/MESOS-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597776#comment-14597776 ] Martin Tapp commented on MESOS-2374: Seems to affect this line https://github.com/apache/mesos/blob/f13239e14f9a982465e93184ef6e34395eb561f4/src/docker/docker.cpp#L379 volumeConfig = volume.host_path() + ":" + volumeConfig Instead of using volume.host_path, should refer to local sandbox path when relative path. Anyone can submit a patch for this. Thanks > Support relative host paths for container volumes > - > > Key: MESOS-2374 > URL: https://issues.apache.org/jira/browse/MESOS-2374 > Project: Mesos > Issue Type: Improvement > Components: containerization, docker >Affects Versions: 0.21.1 >Reporter: Mike Babineau > > There is no convenient way to mount sandbox subdirectories (such as unpacked > archives from fetched URIs) as container volumes. > While it is possible to access sandbox subdirectories via $MESOS_SANDBOX, > this presumes the container is expecting $MESOS_SANDBOX to be passed in. > Furthermore, it also expects the container already knows the resulting > subdirectory paths. Unfortunately, since the archives are extracted by the > fetcher, operators can not control these paths. Path changes to the extracted > archive must be accompanied by a container image change. > One potential solution: > Add support for relative paths to the containerizer. If the containerizer is > given a relative host path, it simply prepends the sandbox path before > passing it to Docker (or similar). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
