[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298975#comment-15298975 ] Gilbert Song commented on MESOS-4757: - [~idownes], Kevin proposed a solution for host user -> container user around two months ago via mailing list. Could you take a look at it to see whether it may break your cases? Thanks! :) https://docs.google.com/document/d/1ENNJKyPrqqm8OsYV8-dDoHTiRmqtuVbcdzNWj1nURsQ/edit#heading=h.j9cu8f69ljik > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[
https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172869#comment-15172869
]
James Peach commented on MESOS-4757:
That would work for Linux and BSD I think, but not for Darwin. I recommend
against providing low-level APIs like {{setgroups}}. It's really easy to get
this wrong with APIs at this level.
> Mesos containerizer should get uid/gids before pivot_root.
> --
>
> Key: MESOS-4757
> URL: https://issues.apache.org/jira/browse/MESOS-4757
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Currently, we call os::su(user) after pivot_root. This is problematic because
> /etc/passwd and /etc/group might be missing in container's root filesystem.
> We should instead, get the uid/gids before pivot_root, and call
> setuid/setgroups after pivot_root.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172842#comment-15172842 ] Jie Yu commented on MESOS-4757: --- OK, ic. Maybe I can just use a large enough number (e.g., 65536)? I think getting this number for sysconf is the right way. I can easily change that. I guess we need a boarder discussion on whether we should do something like this or not (per your email reply and Ian's comment). > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172836#comment-15172836 ] James Peach commented on MESOS-4757: This only works because you have < 16 groups. > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[
https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172817#comment-15172817
]
Ian Downes commented on MESOS-4757:
---
I skimmed the pull request and it looks reasonable.
[~jieyu] Then we should change the the ownership of the sandbox to match? There
doesn't have to be a mapping in the user/group database to set ownership:
{noformat}
[1500][idownes:~]$ touch foo
[1500][idownes:~]$ sudo chown 1234 foo
[1500][idownes:~]$ cat /etc/passwd | grep 1234
[1500][idownes:~]$ stat -f "%N: %u" foo
foo: 1234
{noformat}
> Mesos containerizer should get uid/gids before pivot_root.
> --
>
> Key: MESOS-4757
> URL: https://issues.apache.org/jira/browse/MESOS-4757
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Currently, we call os::su(user) after pivot_root. This is problematic because
> /etc/passwd and /etc/group might be missing in container's root filesystem.
> We should instead, get the uid/gids before pivot_root, and call
> setuid/setgroups after pivot_root.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172684#comment-15172684 ] Cong Wang commented on MESOS-4757: -- Appc already fixes this by: https://github.com/appc/spec/pull/315/files . Mesos could take the similar approach. > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172479#comment-15172479 ] Jie Yu commented on MESOS-4757: --- [~idownes] My main concern is about the sandbox. Currently, sandbox is prepared by the agent (thus using the agent's host database) when chown happens and we bind mount that directory to the container. Without user namespace, I don't know if using the container database is desired or not. > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172440#comment-15172440 ] Ian Downes commented on MESOS-4757: --- IMHO this is incorrect and highlights the inconsistent relationship we have between the host and the container environments, mostly attributable to our history of running in the host context. Ideally, the container should be completely independent of the host configuration! It should not be resolving user/group names to uids/gids using the host's database. That is making huge assumptions about consistent configuration across a cluster -- and an external system to maintain it -- that are unnecessary and undesirable. I suggest something like the following behavior when container images are used: # If a job specifies a user and group name then the container image *must* include the necessary user and group database files and must resolve the names to ids. If not, then it fails. # Support the job specifying uid and gid(s) directly. # Also support picking the user and gid off a file in the image (I think appc supports this?). If a container image is not used then fallback to the current (and terrible) behavior of using the host's databases. Thoughts? > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[
https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171311#comment-15171311
]
Jie Yu commented on MESOS-4757:
---
BTW, I tested my patch on OSX (EL Capitan, 10.11.3), and it works fine.
{noformat}
$ sudo sbin/mesos-master --work_dir=/tmp/mesos/master
$ sudo GLOG_v=1 sbin/mesos-slave --master=10.0.1.26:5050
--work_dir=/tmp/mesos/slave --executor_environment_variables="{}"
$ bin/mesos-execute --master=10.0.1.26:5050 --name=test --command="id" # under
my name 'jie'
Registered executor on 10.0.1.26
Starting task test
sh -c 'id'
Forked command at 86930
uid=501(jie) gid=20(staff)
groups=20(staff),701(com.apple.sharepoint.group.1),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),33(_appstore),100(_lpoperator),204(_developer),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)
Command exited with status 0 (pid: 86930)
Shutting down
Sending SIGTERM to process tree at pid 86930
Sent SIGTERM to the following process trees:
[
]
$ id
uid=501(jie) gid=20(staff)
groups=20(staff),701(com.apple.sharepoint.group.1),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),33(_appstore),100(_lpoperator),204(_developer),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh)
> Mesos containerizer should get uid/gids before pivot_root.
> --
>
> Key: MESOS-4757
> URL: https://issues.apache.org/jira/browse/MESOS-4757
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Currently, we call os::su(user) after pivot_root. This is problematic because
> /etc/passwd and /etc/group might be missing in container's root filesystem.
> We should instead, get the uid/gids before pivot_root, and call
> setuid/setgroups after pivot_root.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171290#comment-15171290 ] Jie Yu commented on MESOS-4757: --- [~jamespeach] Can you also give me a pointer to the 'setgroups' problem you mentioned on Darwin? > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[ https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171286#comment-15171286 ] Jie Yu commented on MESOS-4757: --- I am not familiar with BSD, is there a way to retain capabilities to do pivot_root when switching the credentials? > Mesos containerizer should get uid/gids before pivot_root. > -- > > Key: MESOS-4757 > URL: https://issues.apache.org/jira/browse/MESOS-4757 > Project: Mesos > Issue Type: Bug >Reporter: Jie Yu >Assignee: Jie Yu > > Currently, we call os::su(user) after pivot_root. This is problematic because > /etc/passwd and /etc/group might be missing in container's root filesystem. > We should instead, get the uid/gids before pivot_root, and call > setuid/setgroups after pivot_root. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4757) Mesos containerizer should get uid/gids before pivot_root.
[
https://issues.apache.org/jira/browse/MESOS-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171267#comment-15171267
]
James Peach commented on MESOS-4757:
I think this is a problematic approach. Switching credentials tends to be a bit
subtle on many systems and it doesn't easily decompose into separate operations.
For example, BSD requires (or assumes) that the first {{setgroups(2)}} element
is the primary GID. {{NGROUPS_MAX}} is a dynamic parameter on many systems. In
Darwin, {{setgroups(2)}} just primes the kernel credential cache, but only if
you call the {{initgroups}} system call afterwards.
I suggest that a more reliable approach is to keep doing a full credential
switch before the {{pivot_root}}, but retain enough capabilities to be able to
enter the chroot afterwards.
> Mesos containerizer should get uid/gids before pivot_root.
> --
>
> Key: MESOS-4757
> URL: https://issues.apache.org/jira/browse/MESOS-4757
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Currently, we call os::su(user) after pivot_root. This is problematic because
> /etc/passwd and /etc/group might be missing in container's root filesystem.
> We should instead, get the uid/gids before pivot_root, and call
> setuid/setgroups after pivot_root.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
