Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
On Thu, 2017-05-18 at 03:29 +, Steven Dake (stdake) wrote: > My experience with BTRFS has been flawless. My experience with > overlayfs is that occasionally (older centos kernels) returned > as permissions (rather the drwxrwrw). This most often > happened after using the yum overlay driver. I’ve found overlay to > be pretty reliable as a “read-only” filesystem – eg just serving up > container images, not persistent storage. We've now switched to 'overlay2' and things seem happier. CI passes and for me locally I'm not seeing any issues in TripleO CI yet either. Curious to see if the Kolla tests upstream work with it as well: https://review.openstack.org/#/c/465920/ Dan > > YMMV. Overlayfs is the long-term filesystem of choice for the use > case you outlined. I’ve heard overlayfs has improved over the last > year in terms of backport quality so maybe it is approaching ready. > > Regards > -steve > > > From: Steve Baker > Reply-To: "OpenStack Development Mailing List (not for usage > questions)" > Date: Wednesday, May 17, 2017 at 7:30 PM > To: "OpenStack Development Mailing List (not for usage questions)" penstack-...@lists.openstack.org>, "dwa...@redhat.com" .com> > Subject: Re: [openstack-dev] [TripleO][Kolla] default docker storage > backend for TripleO > > > > On Thu, May 18, 2017 at 12:38 PM, Fox, Kevin M > wrote: > I've only used btrfs and devicemapper on el7. btrfs has worked well. > devicemapper ate may data on multiple occasions. Is redhat supporting > overlay in the el7 kernels now? > > overlay2 is documented as a Technology Preview graph driver in the > Atomic Host 7.3.4 release notes: > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linu > x_atomic_host/7/html-single/release_notes/ > > > > _________________ > From: Dan Prince [dpri...@redhat.com] > Sent: Wednesday, May 17, 2017 5:24 PM > To: openstack-dev > Subject: [openstack-dev] [TripleO][Kolla] default docker storage > backend for TripleO > > TripleO currently uses the default "loopback" docker storage device. > This is not recommended for production (see 'docker info'). > > We've been poking around with docker storage backends in TripleO for > almost 2 months now here: > > https://review.openstack.org/#/c/451916/ > > For TripleO there are a couple of considerations: > > - we intend to support in place upgrades from baremetal to > containers > > - when doing in place upgrades re-partitioning disks is hard, if not > impossible. This makes using devicemapper hard. > > - we'd like to to use a docker storage backend that is production > ready. > > - our target OS is latest Centos/RHEL 7 > > As we approach pike 2 I'm keen to move towards a more production > docker > storage backend. Is there consensus that 'overlay2' is a reasonable > approach to this? Or is it too early to use that with the > combinations > above? > > Looking around at what is recommended in other projects it seems to > be > a mix as well from devicemapper to btrfs. > > [1] https://docs.openshift.com/container-platform/3.3/install_config/ > in > stall/host_preparation.html#configuring-docker-storage > [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_Re > dH > at.sh#n30 > > > I'd love to be able to use overlay2. I've CCed Daniel Walsh with the > hope we can get a general overview of the maturity of overlay2 on > rhel/centos. > > I tried using overlay2 recently to create an undercloud and hit an > issue doing a "cp -a *" on deleted files. This was with kernel- > 3.10.0-514.16.1 and docker-1.12.6. > > I want to get to the bottom of it so I'll reproduce and raise a bug > as appropriate. > _ > _ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
On 18.05.2017 2:38, Fox, Kevin M wrote: > I've only used btrfs and devicemapper on el7. btrfs has worked well. > devicemapper ate may data on multiple occasions. Is redhat supporting overlay > in the el7 kernels now? Please take a look this fs benchmark results thread and comments [0] before evaluating btrfs: [tl;dr] btrfs performed very slow for some cases. [0] https://news.ycombinator.com/item?id=11749010 > > Thanks, > Kevin > > From: Dan Prince [dpri...@redhat.com] > Sent: Wednesday, May 17, 2017 5:24 PM > To: openstack-dev > Subject: [openstack-dev] [TripleO][Kolla] default docker storage backend for > TripleO > > TripleO currently uses the default "loopback" docker storage device. > This is not recommended for production (see 'docker info'). > > We've been poking around with docker storage backends in TripleO for > almost 2 months now here: > > https://review.openstack.org/#/c/451916/ > > For TripleO there are a couple of considerations: > > - we intend to support in place upgrades from baremetal to containers > > - when doing in place upgrades re-partitioning disks is hard, if not > impossible. This makes using devicemapper hard. > > - we'd like to to use a docker storage backend that is production > ready. > > - our target OS is latest Centos/RHEL 7 > > As we approach pike 2 I'm keen to move towards a more production docker > storage backend. Is there consensus that 'overlay2' is a reasonable > approach to this? Or is it too early to use that with the combinations > above? > > Looking around at what is recommended in other projects it seems to be > a mix as well from devicemapper to btrfs. > > [1] https://docs.openshift.com/container-platform/3.3/install_config/in > stall/host_preparation.html#configuring-docker-storage > [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH > at.sh#n30 > > > Dan > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Best regards, Bogdan Dobrelya, Irc #bogdando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
My experience with BTRFS has been flawless. My experience with overlayfs is that occasionally (older centos kernels) returned as permissions (rather the drwxrwrw). This most often happened after using the yum overlay driver. I’ve found overlay to be pretty reliable as a “read-only” filesystem – eg just serving up container images, not persistent storage. YMMV. Overlayfs is the long-term filesystem of choice for the use case you outlined. I’ve heard overlayfs has improved over the last year in terms of backport quality so maybe it is approaching ready. Regards -steve From: Steve Baker Reply-To: "OpenStack Development Mailing List (not for usage questions)" Date: Wednesday, May 17, 2017 at 7:30 PM To: "OpenStack Development Mailing List (not for usage questions)" , "dwa...@redhat.com" Subject: Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO On Thu, May 18, 2017 at 12:38 PM, Fox, Kevin M mailto:kevin@pnnl.gov>> wrote: I've only used btrfs and devicemapper on el7. btrfs has worked well. devicemapper ate may data on multiple occasions. Is redhat supporting overlay in the el7 kernels now? overlay2 is documented as a Technology Preview graph driver in the Atomic Host 7.3.4 release notes: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html-single/release_notes/ _ From: Dan Prince [dpri...@redhat.com<mailto:dpri...@redhat.com>] Sent: Wednesday, May 17, 2017 5:24 PM To: openstack-dev Subject: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO TripleO currently uses the default "loopback" docker storage device. This is not recommended for production (see 'docker info'). We've been poking around with docker storage backends in TripleO for almost 2 months now here: https://review.openstack.org/#/c/451916/ For TripleO there are a couple of considerations: - we intend to support in place upgrades from baremetal to containers - when doing in place upgrades re-partitioning disks is hard, if not impossible. This makes using devicemapper hard. - we'd like to to use a docker storage backend that is production ready. - our target OS is latest Centos/RHEL 7 As we approach pike 2 I'm keen to move towards a more production docker storage backend. Is there consensus that 'overlay2' is a reasonable approach to this? Or is it too early to use that with the combinations above? Looking around at what is recommended in other projects it seems to be a mix as well from devicemapper to btrfs. [1] https://docs.openshift.com/container-platform/3.3/install_config/in stall/host_preparation.html#configuring-docker-storage [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH at.sh#n30 I'd love to be able to use overlay2. I've CCed Daniel Walsh with the hope we can get a general overview of the maturity of overlay2 on rhel/centos. I tried using overlay2 recently to create an undercloud and hit an issue doing a "cp -a *" on deleted files. This was with kernel-3.10.0-514.16.1 and docker-1.12.6. I want to get to the bottom of it so I'll reproduce and raise a bug as appropriate. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
On Thu, May 18, 2017 at 12:38 PM, Fox, Kevin M wrote: > I've only used btrfs and devicemapper on el7. btrfs has worked well. > devicemapper ate may data on multiple occasions. Is redhat supporting > overlay in the el7 kernels now? > overlay2 is documented as a Technology Preview graph driver in the Atomic Host 7.3.4 release notes: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html-single/release_notes/ > _ > From: Dan Prince [dpri...@redhat.com] > Sent: Wednesday, May 17, 2017 5:24 PM > To: openstack-dev > Subject: [openstack-dev] [TripleO][Kolla] default docker storage backend > forTripleO > > TripleO currently uses the default "loopback" docker storage device. > This is not recommended for production (see 'docker info'). > > We've been poking around with docker storage backends in TripleO for > almost 2 months now here: > > https://review.openstack.org/#/c/451916/ > > For TripleO there are a couple of considerations: > > - we intend to support in place upgrades from baremetal to containers > > - when doing in place upgrades re-partitioning disks is hard, if not > impossible. This makes using devicemapper hard. > > - we'd like to to use a docker storage backend that is production > ready. > > - our target OS is latest Centos/RHEL 7 > > As we approach pike 2 I'm keen to move towards a more production docker > storage backend. Is there consensus that 'overlay2' is a reasonable > approach to this? Or is it too early to use that with the combinations > above? > > Looking around at what is recommended in other projects it seems to be > a mix as well from devicemapper to btrfs. > > [1] https://docs.openshift.com/container-platform/3.3/install_config/in > stall/host_preparation.html#configuring-docker-storage > [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH > at.sh#n30 > > I'd love to be able to use overlay2. I've CCed Daniel Walsh with the hope we can get a general overview of the maturity of overlay2 on rhel/centos. I tried using overlay2 recently to create an undercloud and hit an issue doing a "cp -a *" on deleted files. This was with kernel-3.10.0-514.16.1 and docker-1.12.6. I want to get to the bottom of it so I'll reproduce and raise a bug as appropriate. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
btrfs and direct-lvm is recommended for prod env. overlay is bad. On Thu, May 18, 2017 at 8:38 AM, Fox, Kevin M wrote: > I've only used btrfs and devicemapper on el7. btrfs has worked well. > devicemapper ate may data on multiple occasions. Is redhat supporting > overlay in the el7 kernels now? > > Thanks, > Kevin > > From: Dan Prince [dpri...@redhat.com] > Sent: Wednesday, May 17, 2017 5:24 PM > To: openstack-dev > Subject: [openstack-dev] [TripleO][Kolla] default docker storage backend > forTripleO > > TripleO currently uses the default "loopback" docker storage device. > This is not recommended for production (see 'docker info'). > > We've been poking around with docker storage backends in TripleO for > almost 2 months now here: > > https://review.openstack.org/#/c/451916/ > > For TripleO there are a couple of considerations: > > - we intend to support in place upgrades from baremetal to containers > > - when doing in place upgrades re-partitioning disks is hard, if not > impossible. This makes using devicemapper hard. > > - we'd like to to use a docker storage backend that is production > ready. > > - our target OS is latest Centos/RHEL 7 > > As we approach pike 2 I'm keen to move towards a more production docker > storage backend. Is there consensus that 'overlay2' is a reasonable > approach to this? Or is it too early to use that with the combinations > above? > > Looking around at what is recommended in other projects it seems to be > a mix as well from devicemapper to btrfs. > > [1] https://docs.openshift.com/container-platform/3.3/install_config/in > stall/host_preparation.html#configuring-docker-storage > [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH > at.sh#n30 > > > Dan > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Jeffrey Zhang Blog: http://xcodest.me __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
I've only used btrfs and devicemapper on el7. btrfs has worked well. devicemapper ate may data on multiple occasions. Is redhat supporting overlay in the el7 kernels now? Thanks, Kevin From: Dan Prince [dpri...@redhat.com] Sent: Wednesday, May 17, 2017 5:24 PM To: openstack-dev Subject: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO TripleO currently uses the default "loopback" docker storage device. This is not recommended for production (see 'docker info'). We've been poking around with docker storage backends in TripleO for almost 2 months now here: https://review.openstack.org/#/c/451916/ For TripleO there are a couple of considerations: - we intend to support in place upgrades from baremetal to containers - when doing in place upgrades re-partitioning disks is hard, if not impossible. This makes using devicemapper hard. - we'd like to to use a docker storage backend that is production ready. - our target OS is latest Centos/RHEL 7 As we approach pike 2 I'm keen to move towards a more production docker storage backend. Is there consensus that 'overlay2' is a reasonable approach to this? Or is it too early to use that with the combinations above? Looking around at what is recommended in other projects it seems to be a mix as well from devicemapper to btrfs. [1] https://docs.openshift.com/container-platform/3.3/install_config/in stall/host_preparation.html#configuring-docker-storage [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH at.sh#n30 Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
Be careful with overlay, I've seen it acting in a ways you don't want it to act up. That was some time ago, but memories persist. To my experience best option is btrfs. If you don't want to repartition disk, btrfs on loopback isn't horrible too. deviemapper on loopback is horrible, but that's different. On 17 May 2017 at 17:24, Dan Prince wrote: > TripleO currently uses the default "loopback" docker storage device. > This is not recommended for production (see 'docker info'). > > We've been poking around with docker storage backends in TripleO for > almost 2 months now here: > > https://review.openstack.org/#/c/451916/ > > For TripleO there are a couple of considerations: > > - we intend to support in place upgrades from baremetal to containers > > - when doing in place upgrades re-partitioning disks is hard, if not > impossible. This makes using devicemapper hard. > > - we'd like to to use a docker storage backend that is production > ready. > > - our target OS is latest Centos/RHEL 7 > > As we approach pike 2 I'm keen to move towards a more production docker > storage backend. Is there consensus that 'overlay2' is a reasonable > approach to this? Or is it too early to use that with the combinations > above? > > Looking around at what is recommended in other projects it seems to be > a mix as well from devicemapper to btrfs. > > [1] https://docs.openshift.com/container-platform/3.3/install_config/in > stall/host_preparation.html#configuring-docker-storage > [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH > at.sh#n30 > > > Dan > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO][Kolla] default docker storage backend for TripleO
TripleO currently uses the default "loopback" docker storage device. This is not recommended for production (see 'docker info'). We've been poking around with docker storage backends in TripleO for almost 2 months now here: https://review.openstack.org/#/c/451916/ For TripleO there are a couple of considerations: - we intend to support in place upgrades from baremetal to containers - when doing in place upgrades re-partitioning disks is hard, if not impossible. This makes using devicemapper hard. - we'd like to to use a docker storage backend that is production ready. - our target OS is latest Centos/RHEL 7 As we approach pike 2 I'm keen to move towards a more production docker storage backend. Is there consensus that 'overlay2' is a reasonable approach to this? Or is it too early to use that with the combinations above? Looking around at what is recommended in other projects it seems to be a mix as well from devicemapper to btrfs. [1] https://docs.openshift.com/container-platform/3.3/install_config/in stall/host_preparation.html#configuring-docker-storage [2] http://git.openstack.org/cgit/openstack/kolla/tree/tools/setup_RedH at.sh#n30 Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
