Hi guys, I was wondering if you have any updates on this. Thanks for your help.
On Sun, Aug 14, 2016 at 7:22 AM, Cory Keane <[email protected]> wrote: > The docker command I try running takes the format: `docker run --rm > --name=blahblahblah -e APP_ENV=production quay.io/path/tocontainer:stable > php aphpfile.php` > > Below you will find the following: > 1. CoreOS Info > 2. Output from running `mount` > > > 1. CoreOS Info: > > NAME=CoreOS > > ID=coreos > > VERSION=1081.5.0 > > VERSION_ID=1081.5.0 > > BUILD_ID=2016-07-16-2255 > > PRETTY_NAME="CoreOS 1081.5.0 (MoreOS)" > > ANSI_COLOR="1;32" > > HOME_URL="https://coreos.com/"; > > BUG_REPORT_URL="https://github.com/coreos/bugs/issues"; > > > Output from running `mount`: > > sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) > > proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) > > devtmpfs on /dev type devtmpfs (rw,nosuid,size=2008876k,nr_ > inodes=502219,mode=755) > > securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec, > relatime) > > tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) > > devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime, > gid=5,mode=620,ptmxmode=000) > > tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) > > tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) > > cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec, > relatime,xattr,release_agent=/usr/lib/systemd/systemd- > cgroups-agent,name=systemd) > > pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) > > cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec, > relatime,hugetlb) > > cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec, > relatime,freezer) > > cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec, > relatime,cpu,cpuacct) > > cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec, > relatime,blkio) > > cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec, > relatime,memory) > > cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec, > relatime,devices) > > cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup > (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) > > cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec, > relatime,perf_event) > > cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec, > relatime,cpuset) > > cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec, > relatime,pids) > > /dev/xvda9 on / type ext4 (rw,relatime,data=ordered) > > /dev/xvda3 on /usr type ext4 (ro,relatime,block_validity, > delalloc,barrier,user_xattr,acl) > > selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime) > > systemd-1 on /proc/sys/fs/binfmt_misc type autofs > (rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto= > 5,direct,pipe_ino=11137) > > tmpfs on /tmp type tmpfs (rw) > > tmpfs on /media type tmpfs (rw,nosuid,nodev,noexec,relatime) > > hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime) > > debugfs on /sys/kernel/debug type debugfs (rw,relatime) > > xenfs on /proc/xen type xenfs (rw,relatime) > > systemd-1 on /boot type autofs (rw,relatime,fd=35,pgrp=1, > timeout=0,minproto=5,maxproto=5,direct,pipe_ino=11461) > > mqueue on /dev/mqueue type mqueue (rw,relatime) > > /dev/xvda6 on /usr/share/oem type ext4 (rw,nodev,relatime,commit=600, > data=ordered) > > /dev/xvda1 on /boot type vfat (rw,relatime,fmask=0022,dmask= > 0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro) > > tmpfs on /run/user/500 type tmpfs (rw,nosuid,nodev,relatime, > size=404996k,mode=700,uid=500,gid=500) > > > Thank you, > > Cory > > On Sat, Aug 13, 2016 at 9:29 PM, Brandon Philips < > [email protected]> wrote: > >> Hello Cory- >> >> Also, what versions of CoreOS is this happening under? `cat >> /etc/os-release`. >> >> Brandon >> >> >> On Sat, Aug 13, 2016 at 7:26 PM Brandon Philips < >> [email protected]> wrote: >> >>> Hello Cory- >>> >>> There are a number of errors about the mounts which should be non-fatal: >>> >>> Aug 09 15:40:08 ip-172-31-16-118.ec2.internal dockerd[1107]: >>> time="2016-08-09T15:40:08.520711349Z" level=error msg="Clean up Error! >>> Cannot destroy container 26537dcd69eeb25dcd3a4867f81db9 >>> 09e5ca992efa1e6861d9e804770ec11239: rmdriverfs: Driver overlay failed >>> to remove root filesystem 26537dcd69eeb25dcd3a4867f81db9 >>> 09e5ca992efa1e6861d9e804770ec11239: mount still active" >>> Aug 09 15:40:08 ip-172-31-16-118.ec2.internal dockerd[1107]: >>> time="2016-08-09T15:40:08.520749851Z" level=error msg="Handler for POST >>> /v1.22/containers/create returned error: Error relabeling upper directory: >>> operation not supported" >>> >>> Could you grab the output of `mount` from these machines? >>> >>> And it doesn't look like the daemon dies for several minutes after: >>> >>> Aug 09 15:43:41 ip-172-31-16-118.ec2.internal systemd[1]: Stopping >>> Docker Application Container Engine... >>> >>> I really don't know what is getting messed up to cause this. What are >>> the arguments to docker run? >>> >>> It looks sort of a like a repeat of https://github.com/coreos/b >>> ugs/issues/1301 which was about SELinux and OverlayFS so I am going to >>> cc in Nick, Matthew, and Crawford. >>> >>> Cheers, >>> >>> Brandon >>> >>> On Tue, Aug 9, 2016 at 10:41 AM Cory Keane <[email protected]> wrote: >>> >>>> Hey Brandon, >>>> >>>> I decided to just email the logs directly to you. >>>> >>>> Thanks! >>>> >>>> On Tue, Aug 9, 2016 at 12:37 PM, Brandon Philips < >>>> [email protected]> wrote: >>>> >>>>> journalctl --no-pager -u docker.service > foo.log >>>>> >>>>> I think it turns on no-pager by default if it knows stdout isn't a tty >>>>> >>>>> On Tue, Aug 9, 2016 at 10:35 AM Cory Keane <[email protected]> >>>>> wrote: >>>>> >>>>>> No worries Brandon. I will upload them. But what's the easiest way to >>>>>> export them? The command you told me to run makes it so I have to press >>>>>> the >>>>>> space bar to cycle through all the lines of the file. >>>>>> >>>>>> Thanks so much. >>>>>> >>>>>> On Tue, Aug 9, 2016 at 12:34 PM, Brandon Philips < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hey Cory- >>>>>>> >>>>>>> It is really hard to know what to look for in the logs. If you don't >>>>>>> want to share the logs publicly you can email them to me personally and >>>>>>> I >>>>>>> will take a look. >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Brandon >>>>>>> >>>>>>> On Tue, Aug 9, 2016 at 8:42 AM Cory Keane <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Brandon, I apologize. It appears that this worked for one of the >>>>>>>> machines experiencing this issue, but not the other machine. Since I >>>>>>>> posted >>>>>>>> this, another one of our fleet machines auto updated to what appears >>>>>>>> to be >>>>>>>> this faulty version. >>>>>>>> >>>>>>>> So please let me know what I am looking for specifically when >>>>>>>> running the `journalctl -u docker` command. >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Cory Keane* >>>>>> Co-Founder & CTO >>>>>> ThreadMeUp.com >>>>>> >>>>> >>>> >>>> >>>> -- >>>> *Cory Keane* >>>> Co-Founder & CTO >>>> ThreadMeUp.com >>>> >>> > > > -- > *Cory Keane* > Co-Founder & CTO > ThreadMeUp.com > -- *Cory Keane* Co-Founder & CTO ThreadMeUp.com
