openshift.io/scc is "restricted" for app1-45-3blnd (not running as root). It also has that value for app5-36-2rfsq (running as root).
On Mon, Feb 6, 2017 at 1:11 PM, Clayton Coleman <ccole...@redhat.com> wrote: > Were those apps created in order? Or at individual times? If you did > the following order of actions: > > 1. create app2, app4 > 2. grant the default service account access to a higher level SCC > 3. create app1, app3, app5, and app6 > > Then this would be what I would expect. Can you look at the annotations > of pod app1-45-3blnd and see what the value of "openshift.io/scc" is? > > > On Mon, Feb 6, 2017 at 1:57 PM, Alex Wauck <alexwa...@exosite.com> wrote: > >> OK, this just got a lot more interesting: >> >> $ oc -n some-project exec app1-45-3blnd -- id >> uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon >> ),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout), >> 26(tape),27(video),1000370000 >> $ oc -n some-project exec app2-18-q2fwm -- id >> uid=1000370000 gid=0(root) groups=1000370000 >> $ oc -n some-project exec app3-10-lhato -- id >> uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon >> ),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout), >> 26(tape),27(video),1000370000 >> $ oc -n some-project exec app4-16-dl2r7 -- id >> uid=1000370000 gid=0(root) groups=1000370000 >> $ oc -n some-project exec app5-36-2rfsq -- id >> uid=0(root) gid=0(root) groups=0(root),1000370000 >> $ oc -n some-project exec app6-15-078fd -- id >> uid=0(root) gid=0(root) groups=0(root),1000370000 >> >> All of these pods are running on the same node, and as you can see, they >> are in the same project. Yet, some are running as root and some are not. >> How weird is that? >> >> On Mon, Feb 6, 2017 at 12:49 PM, Alex Wauck <alexwa...@exosite.com> >> wrote: >> >>> $ oc export -n some-project pod/good-pod | grep serviceAccount >>> serviceAccount: default >>> serviceAccountName: default >>> $ oc export -n some-project pod/bad-pod | grep serviceAccount >>> serviceAccount: default >>> serviceAccountName: default >>> >>> Same serviceAccountName. This problem seems to happen with any pod from >>> any project that happens to run on these newer nodes. I examined the >>> output of `oc describe scc`, and I did not find any unexpected access to >>> elevated privileges for a default serviceaccount. The project were I'm >>> currently seeing the problem is not mentioned at all. Also, I've seen the >>> problem happen with pods that are managed by the same replication >>> controller. >>> >>> On Mon, Feb 6, 2017 at 12:46 PM, Clayton Coleman <ccole...@redhat.com> >>> wrote: >>> >>>> Adding the list back >>>> >>>> ---------- Forwarded message ---------- >>>> From: Clayton Coleman <ccole...@redhat.com> >>>> Date: Mon, Feb 6, 2017 at 1:42 PM >>>> Subject: Re: Pods randomly running as root >>>> To: Alex Wauck <alexwa...@exosite.com> >>>> Cc: users <us...@redhat.com> >>>> >>>> >>>> Do the pods running as root and not have the same serviceAccountName >>>> field or different ones? IF different, you may have granted the service >>>> account access to a higher role - defaulting is determined by the SCC's >>>> that a service account can access, so an admin level service account will >>>> run as root by default unless you specify you don't want that. >>>> >>>> On Mon, Feb 6, 2017 at 1:37 PM, Alex Wauck <alexwa...@exosite.com> >>>> wrote: >>>> >>>>> I'm looking at two nodes where one has the problem and the other >>>>> doesn't, and I have confirmed that their node-config.yaml is the same for >>>>> both (modulo IP addresses). The generated kubeconfigs for these nodes on >>>>> the master are also the same (modulo IP addresses and keys/certs). >>>>> >>>>> On Mon, Feb 6, 2017 at 10:46 AM, Alex Wauck <alexwa...@exosite.com> >>>>> wrote: >>>>> >>>>>> Oh, wait. I was looking at the wrong section. The non-root pod as a >>>>>> runAsUser attribute, but the root pod doesn't! >>>>>> >>>>>> On Mon, Feb 6, 2017 at 10:44 AM, Alex Wauck <alexwa...@exosite.com> >>>>>> wrote: >>>>>> >>>>>>> A pod that IS running as root have this: >>>>>>> >>>>>>> securityContext: >>>>>>> fsGroup: 1000370000 >>>>>>> seLinuxOptions: >>>>>>> level: s0:c19,c14 >>>>>>> >>>>>>> Another pod in the same project that is NOT running as root has the >>>>>>> exact same securityContext section. >>>>>>> >>>>>>> On Mon, Feb 6, 2017 at 10:25 AM, Clayton Coleman < >>>>>>> ccole...@redhat.com> wrote: >>>>>>> >>>>>>>> Do the pods themselves have a user UID set on them? Each pod >>>>>>>> should have the container "securityContext" field set and have an >>>>>>>> explicit >>>>>>>> user ID value set. >>>>>>>> >>>>>>>> On Mon, Feb 6, 2017 at 11:23 AM, Alex Wauck <alexwa...@exosite.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> These are completely normal app containers. They are managed by >>>>>>>>> deploy configs. Whether they run as root or not seems to depend on >>>>>>>>> which >>>>>>>>> node they run on: the older nodes seem to run pods as random UIDs, >>>>>>>>> while >>>>>>>>> the newer ones run as root. Our older nodes have >>>>>>>>> docker-selinux-1.10.3 >>>>>>>>> installed, while the newer ones do not. They only have >>>>>>>>> docker-selinux-1.9.1 available, since the 1.10.3 package seems to >>>>>>>>> have been >>>>>>>>> removed from the CentOS extras repo. >>>>>>>>> >>>>>>>>> We are running OpenShift 1.2.1, since I haven't had time to >>>>>>>>> upgrade it. >>>>>>>>> >>>>>>>>> On Mon, Feb 6, 2017 at 8:31 AM, Clayton Coleman < >>>>>>>>> ccole...@redhat.com> wrote: >>>>>>>>> >>>>>>>>>> Are you running them directly (launching a pod)? Or running them >>>>>>>>>> under another controller resource. >>>>>>>>>> >>>>>>>>>> On Feb 6, 2017, at 2:00 AM, Alex Wauck <alexwa...@exosite.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Recently, I began to notice that some of my pods on OpenShift run >>>>>>>>>> as root instead of a random UID. There does not seem to be any >>>>>>>>>> obvious >>>>>>>>>> cause (e.g. SCC). Any idea how this could happen or where to look >>>>>>>>>> for >>>>>>>>>> clues? >>>>>>>>>> >>>>>>>>>> >>>> >>> >>> >>> -- >>> >>> Alex Wauck // DevOps Engineer >>> >>> *E X O S I T E* >>> *www.exosite.com <http://www.exosite.com/>* >>> >>> Making Machines More Human. >>> >>> >> >> >> -- >> >> Alex Wauck // DevOps Engineer >> >> *E X O S I T E* >> *www.exosite.com <http://www.exosite.com/>* >> >> Making Machines More Human. >> >> > -- Alex Wauck // DevOps Engineer *E X O S I T E* *www.exosite.com <http://www.exosite.com/>* Making Machines More Human.
_______________________________________________ users mailing list users@lists.openshift.redhat.com http://lists.openshift.redhat.com/openshiftmm/listinfo/users
