Hello Cory, I can't find any issues relating specifically to the "docker: Error response from daemon: Error relabeling upper directory: operation not supported." error. If you could update to the latest beta and see if this error continues before submitting an issue to coreos/bugs <https://github.com/coreos/bugs> that would be ideal.
Cheers, Kyle Brown On Thu, Sep 1, 2016 at 9:07 AM, Cory Keane <[email protected]> wrote: > Hey guys, > > Is this an open issue? > > On Mon, Aug 22, 2016 at 9:27 AM, Cory Keane <[email protected]> wrote: > >> Hi guys, >> >> I was wondering if you have any updates on this. Thanks for your help. >> >> On Sun, Aug 14, 2016 at 7:22 AM, Cory Keane <[email protected]> wrote: >> >>> The docker command I try running takes the format: `docker run --rm >>> --name=blahblahblah -e APP_ENV=production quay.io/path/tocontainer:stabl >>> e php aphpfile.php` >>> >>> Below you will find the following: >>> 1. CoreOS Info >>> 2. Output from running `mount` >>> >>> >>> 1. CoreOS Info: >>> >>> NAME=CoreOS >>> >>> ID=coreos >>> >>> VERSION=1081.5.0 >>> >>> VERSION_ID=1081.5.0 >>> >>> BUILD_ID=2016-07-16-2255 >>> >>> PRETTY_NAME="CoreOS 1081.5.0 (MoreOS)" >>> >>> ANSI_COLOR="1;32" >>> >>> HOME_URL="https://coreos.com/"; >>> >>> BUG_REPORT_URL="https://github.com/coreos/bugs/issues"; >>> >>> >>> Output from running `mount`: >>> >>> sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) >>> >>> proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) >>> >>> devtmpfs on /dev type devtmpfs (rw,nosuid,size=2008876k,nr_in >>> odes=502219,mode=755) >>> >>> securityfs on /sys/kernel/security type securityfs >>> (rw,nosuid,nodev,noexec,relatime) >>> >>> tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) >>> >>> devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid >>> =5,mode=620,ptmxmode=000) >>> >>> tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) >>> >>> tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) >>> >>> cgroup on /sys/fs/cgroup/systemd type cgroup >>> (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/li >>> b/systemd/systemd-cgroups-agent,name=systemd) >>> >>> pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) >>> >>> cgroup on /sys/fs/cgroup/hugetlb type cgroup >>> (rw,nosuid,nodev,noexec,relatime,hugetlb) >>> >>> cgroup on /sys/fs/cgroup/freezer type cgroup >>> (rw,nosuid,nodev,noexec,relatime,freezer) >>> >>> cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup >>> (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) >>> >>> cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relati >>> me,blkio) >>> >>> cgroup on /sys/fs/cgroup/memory type cgroup >>> (rw,nosuid,nodev,noexec,relatime,memory) >>> >>> cgroup on /sys/fs/cgroup/devices type cgroup >>> (rw,nosuid,nodev,noexec,relatime,devices) >>> >>> cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup >>> (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) >>> >>> cgroup on /sys/fs/cgroup/perf_event type cgroup >>> (rw,nosuid,nodev,noexec,relatime,perf_event) >>> >>> cgroup on /sys/fs/cgroup/cpuset type cgroup >>> (rw,nosuid,nodev,noexec,relatime,cpuset) >>> >>> cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relati >>> me,pids) >>> >>> /dev/xvda9 on / type ext4 (rw,relatime,data=ordered) >>> >>> /dev/xvda3 on /usr type ext4 (ro,relatime,block_validity,de >>> lalloc,barrier,user_xattr,acl) >>> >>> selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime) >>> >>> systemd-1 on /proc/sys/fs/binfmt_misc type autofs >>> (rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,di >>> rect,pipe_ino=11137) >>> >>> tmpfs on /tmp type tmpfs (rw) >>> >>> tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime) >>> >>> hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) >>> >>> debugfs on /sys/kernel/debug type debugfs (rw,relatime) >>> >>> xenfs on /proc/xen type xenfs (rw,relatime) >>> >>> systemd-1 on /boot type autofs (rw,relatime,fd=35,pgrp=1,time >>> out=0,minproto=5,maxproto=5,direct,pipe_ino=11461) >>> >>> mqueue on /dev/mqueue type mqueue (rw,relatime) >>> >>> /dev/xvda6 on /usr/share/oem type ext4 (rw,nodev,relatime,commit=600, >>> data=ordered) >>> >>> /dev/xvda1 on /boot type vfat (rw,relatime,fmask=0022,dmask= >>> 0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro) >>> >>> tmpfs on /run/user/500 type tmpfs (rw,nosuid,nodev,relatime,size >>> =404996k,mode=700,uid=500,gid=500) >>> >>> >>> Thank you, >>> >>> Cory >>> >>> On Sat, Aug 13, 2016 at 9:29 PM, Brandon Philips < >>> [email protected]> wrote: >>> >>>> Hello Cory- >>>> >>>> Also, what versions of CoreOS is this happening under? `cat >>>> /etc/os-release`. >>>> >>>> Brandon >>>> >>>> >>>> On Sat, Aug 13, 2016 at 7:26 PM Brandon Philips < >>>> [email protected]> wrote: >>>> >>>>> Hello Cory- >>>>> >>>>> There are a number of errors about the mounts which should be >>>>> non-fatal: >>>>> >>>>> Aug 09 15:40:08 ip-172-31-16-118.ec2.internal dockerd[1107]: >>>>> time="2016-08-09T15:40:08.520711349Z" level=error msg="Clean up >>>>> Error! Cannot destroy container 26537dcd69eeb25dcd3a4867f81db9 >>>>> 09e5ca992efa1e6861d9e804770ec11239: rmdriverfs: Driver overlay failed >>>>> to remove root filesystem 26537dcd69eeb25dcd3a4867f81db9 >>>>> 09e5ca992efa1e6861d9e804770ec11239: mount still active" >>>>> Aug 09 15:40:08 ip-172-31-16-118.ec2.internal dockerd[1107]: >>>>> time="2016-08-09T15:40:08.520749851Z" level=error msg="Handler for >>>>> POST /v1.22/containers/create returned error: Error relabeling upper >>>>> directory: operation not supported" >>>>> >>>>> Could you grab the output of `mount` from these machines? >>>>> >>>>> And it doesn't look like the daemon dies for several minutes after: >>>>> >>>>> Aug 09 15:43:41 ip-172-31-16-118.ec2.internal systemd[1]: Stopping >>>>> Docker Application Container Engine... >>>>> >>>>> I really don't know what is getting messed up to cause this. What are >>>>> the arguments to docker run? >>>>> >>>>> It looks sort of a like a repeat of https://github.com/coreos/b >>>>> ugs/issues/1301 which was about SELinux and OverlayFS so I am going >>>>> to cc in Nick, Matthew, and Crawford. >>>>> >>>>> Cheers, >>>>> >>>>> Brandon >>>>> >>>>> On Tue, Aug 9, 2016 at 10:41 AM Cory Keane <[email protected]> >>>>> wrote: >>>>> >>>>>> Hey Brandon, >>>>>> >>>>>> I decided to just email the logs directly to you. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> On Tue, Aug 9, 2016 at 12:37 PM, Brandon Philips < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> journalctl --no-pager -u docker.service > foo.log >>>>>>> >>>>>>> I think it turns on no-pager by default if it knows stdout isn't a >>>>>>> tty >>>>>>> >>>>>>> On Tue, Aug 9, 2016 at 10:35 AM Cory Keane <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> No worries Brandon. I will upload them. But what's the easiest way >>>>>>>> to export them? The command you told me to run makes it so I have to >>>>>>>> press >>>>>>>> the space bar to cycle through all the lines of the file. >>>>>>>> >>>>>>>> Thanks so much. >>>>>>>> >>>>>>>> On Tue, Aug 9, 2016 at 12:34 PM, Brandon Philips < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hey Cory- >>>>>>>>> >>>>>>>>> It is really hard to know what to look for in the logs. If you >>>>>>>>> don't want to share the logs publicly you can email them to me >>>>>>>>> personally >>>>>>>>> and I will take a look. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Brandon >>>>>>>>> >>>>>>>>> On Tue, Aug 9, 2016 at 8:42 AM Cory Keane <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Brandon, I apologize. It appears that this worked for one of the >>>>>>>>>> machines experiencing this issue, but not the other machine. Since I >>>>>>>>>> posted >>>>>>>>>> this, another one of our fleet machines auto updated to what appears >>>>>>>>>> to be >>>>>>>>>> this faulty version. >>>>>>>>>> >>>>>>>>>> So please let me know what I am looking for specifically when >>>>>>>>>> running the `journalctl -u docker` command. >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Cory Keane* >>>>>>>> Co-Founder & CTO >>>>>>>> ThreadMeUp.com >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Cory Keane* >>>>>> Co-Founder & CTO >>>>>> ThreadMeUp.com >>>>>> >>>>> >>> >>> >>> -- >>> *Cory Keane* >>> Co-Founder & CTO >>> ThreadMeUp.com >>> >> >> >> >> -- >> *Cory Keane* >> Co-Founder & CTO >> ThreadMeUp.com >> > > > > -- > *Cory Keane* > Co-Founder & CTO > ThreadMeUp.com >
