While Seth's workaround allows Kubernetes containers to read /etc/hosts, I've discovered a similar problem with /dev/shm. A container started with "docker run" can access /dev/shm normally, but one started by Kubernetes cannot. After carefully comparing the output from "docker inspect" for the two situations, I discovered that Kubernetes is using Docker's IPC mechanism to share /dev/shm between two containers in a pod. It seems that while Docker would normally set SELinux labels to allow both containers to read and write, Kubernetes overrides SELinux labels, disallowing access for all but the source container to /dev/shm.
My container needs to use /dev/shm for IPC within the container only. I'm only specifying one container per pod so the container which is the source for /dev/shm must be automatically generated by Kubernetes (it has a name starting with "k8s_POD"). I don't currently need any of the fancy cross-container IPC stuff but I guess for now, I'll just disable SELinux enforcement. Is this a bug in Kubernetes only? Is there a way to work around this, such as disabling any attempt at cross-container IPC? I found this bug which seems to describe exactly the problem I'm having: https://github.com/kubernetes/kubernetes/issues/20899 On 09/08/2016 11:25 AM, Seth Jennings wrote: > Hey Jonathan, > > Yes this is an issue. > > kube mounts /etc/hosts into the container from something like: > > /var/lib/kubelet/pods/9679b088-75d6-11e6-9f2b-fa163eebc035/etc-host > > on the host. That file is currently labeled with docker_var_lib_t: > > # ls -alZ etc-hosts > -rw-r--r--. root root system_u:object_r:docker_var_lib_t:s0 etc-hosts > > This should be resolved by this yet-to-be-released fix to selinux-policy: > > https://bugzilla.redhat.com/show_bug.cgi?id=1369159 > > Should be fixed in selinux-policy > 3.13.1-95.el7. > > In the meantime, you can override the selinux policy on your system manually: > > # semanage fcontext -l | grep kubelet > /var/lib/kubelet(/.*)? all files > system_u:object_r:docker_var_lib_t:s0 > > # semanage fcontext -a -t svirt_sandbox_file_t "/var/lib/kubelet(/.*)?" > > # restorecon -Rv /var/lib/kubelet > > Hope this helps! > > Seth > > On Sun, Sep 4, 2016 at 8:15 PM, Jonathan Rogers > <[email protected]> wrote: >> I'm using CentOS Atomic Host 7. I can run Docker containers directly and >> they function as expected. I set up a small Kubernetes cluster using the >> Atomic and Kubernetes documentation. I can also run containers using >> Kubernetes, but /etc/hosts is unreadable in the containers because of >> SELinux configuration. I found this be true in containers based on both >> the "busybox" Docker image as well as my CentOS 6-based image. >> >> I see that Kubernetes sets up a mount just for /etc/hosts, overriding >> Docker's default behavior. Why is this necessary? It seems that >> Kubernetes fails to apply the necessary label(s) to the hosts file it >> provides. If I use the chcon command on the host to add >> "svirt_sandbox_file_t" to the Kubernetes-managed hosts file, the >> container can read it via /etc/hosts. Of course, disabling SELinux >> enforcement also avoids the problem. >> >> Since this doesn't seem to be a common problem, I can't tell if it's >> Kubernetes, CentOS or Atomic at fault. I found a long discussion about >> SELinux vs Kubernetes which seems related. However, the Docker volume in >> question was generated automatically by Kubernetes without any explicit >> configuration. >> >> https://github.com/projectatomic/adb-atomic-developer-bundle/issues/117 >> >> -- >> Jonathan Rogers >> Socialserve.com by Emphasys Software >> [email protected] >> >> > -- Jonathan Rogers Socialserve.com by Emphasys Software [email protected]
