Hi all, If we want to run etcd and flannel in container, we will introduce docker-bootstrap which makes setup become more complex as Egor pointed out. Should we pay for the price?
On Sat, Nov 28, 2015 at 8:45 AM, Egor Guz <e...@walmartlabs.com> wrote: > Wanghua, > > I don’t think moving flannel to the container is good idea. This is setup > great for dev environment, but become too complex from operator point of > view (you add extra Docker daemon and need extra Cinder volume for this > daemon, also > keep in mind it makes sense to keep etcd data folder at Cinder storage as > well because etcd is database). flannel has just there files without extra > dependencies and it’s much easy to download it during cloud-init ;) > > I agree that we have pain with building Fedora Atomic images, but instead > of simplify this process we should switch to another more “friendly” images > (e.g. Fedora/CentOS/Ubuntu) which we can easy build with disk builder. > Also we can fix CoreOS template (I believe people more asked about it > instead of Atomic), but we may face similar to Atomic issues when we will > try to integrate not CoreOS products (e.g. Calico or Weave) > > — > Egor > > From: 王华 <wanghua.hum...@gmail.com<mailto:wanghua.hum...@gmail.com>> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Date: Thursday, November 26, 2015 at 00:15 > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Subject: Re: [openstack-dev] [magnum]storage for docker-bootstrap > > Hi Hongbin, > > The docker in master node stores data in /dev/mapper/atomicos-docker--data > and metadata in /dev/mapper/atomicos-docker--meta. > /dev/mapper/atomicos-docker--data and /dev/mapper/atomicos-docker--meta are > logic volumes. The docker in minion node store data in the cinder volume, > but /dev/mapper/atomicos-docker--meta and /dev/mapper/atomicos-docker--meta > are not used. If we want to leverage Cinder volume for docker in master, > should we drop /dev/mapper/atomicos-docker--meta and > /dev/mapper/atomicos-docker--meta? I think it is not necessary to allocate > a Cinder volume. It is enough to allocate two logic volumes for docker, > because only etcd, flannel, k8s run in the docker daemon which need not a > large amount of storage. > > Best regards, > Wanghua > > On Thu, Nov 26, 2015 at 12:40 AM, Hongbin Lu <hongbin...@huawei.com > <mailto:hongbin...@huawei.com>> wrote: > Here is a bit more context. > > Currently, at k8s and swarm bay, some required binaries (i.e. etcd and > flannel) are built into image and run at host. We are exploring the > possibility to containerize some of these system components. The rationales > are (i) it is infeasible to build custom packages into an atomic image and > (ii) it is infeasible to upgrade individual component. For example, if > there is a bug in current version of flannel and we know the bug was fixed > in the next version, we need to upgrade flannel by building a new image, > which is a tedious process. > > To containerize flannel, we need a second docker daemon, called > docker-bootstrap [1]. In this setup, pods are running on the main docker > daemon, and flannel and etcd are running on the second docker daemon. The > reason is that flannel needs to manage the network of the main docker > daemon, so it needs to run on a separated daemon. > > Daneyon, I think it requires separated storage because it needs to run a > separated docker daemon (unless there is a way to make two docker daemons > share the same storage). > > Wanghua, is it possible to leverage Cinder volume for that. Leveraging > external storage is always preferred [2]. > > [1] > http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode.html#bootstrap-docker > [2] http://www.projectatomic.io/docs/docker-storage-recommendation/ > > Best regards, > Hongbin > > From: Daneyon Hansen (danehans) [mailto:daneh...@cisco.com<mailto: > daneh...@cisco.com>] > Sent: November-25-15 11:10 AM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [magnum]storage for docker-bootstrap > > > > From: 王华 <wanghua.hum...@gmail.com<mailto:wanghua.hum...@gmail.com>> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Date: Wednesday, November 25, 2015 at 5:00 AM > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org > >> > Subject: [openstack-dev] [magnum]storage for docker-bootstrap > > Hi all, > > I am working on containerizing etcd and flannel. But I met a problem. As > described in [1], we need a docker-bootstrap. Docker and docker-bootstrap > can not use the same storage, so we need some disk space for it. > > I reviewed [1] and I do not see where the bootstrap docker instance > requires separate storage. > > The docker in master node stores data in /dev/mapper/atomicos-docker--data > and metadata in /dev/mapper/atomicos-docker--meta. The disk space left is > too same for docker-bootstrap. Even if the root_gb of the instance flavor > is 20G, only 8G can be used in our image. I want to make it bigger. One way > is we can add the disk space left in the vda as vda3 into atomicos vg after > the instance starts and we allocate two logic volumes for docker-bootstrap. > Another way is when we create the image, we allocate two logic volumes for > docker-bootstrap. The second way has a advantage. It doesn't have to make > filesystem when the instance is created which is time consuming. > > What is your opinion? > > Best Regards > Wanghua > > [1] > http://kubernetes.io/v1.1/docs/getting-started-guides/docker-multinode/master.html > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev