One benefit of the kolla API that I've not seen mentioned yet (sorry if I missed it) is that you can change files on the host without affecting the running container. Bind mounts don't have this property. This is handy for reconfiguration/upgrade operations, where we write out a new set of config before recreating/restarting the container. COPY_ONCE is the king of immutable here, but even for COPY_ALWAYS, this works as long as the container doesn't restart while the config files are being written.
Mark On 5 April 2018 at 21:41, Michał Jastrzębski <inc...@gmail.com> wrote: > So I'll re-iterate comment which I made in BCN. In previous thread we > praised how Kolla provided stable API for images, and I agree that it > was great design choice (to provide stable API, not necessarily how > API looks), and this change would break it. So *if* we decide to do > it, we need to follow deprecation, that means we could deprecate these > files in this release and start removing them in next. > > Support for LOCI in kolla-ansible is good thing, but I don't think > changing Kolla image API is required for that. LOCI provides base > image arument, so we could simply create base-image with all the > extended-start and set-config mechanisms and some shim to source > extended-start script that belongs to particular container. We will > need kolla layer image anyway because set_config is there to stay (as > Martin pointed out it's valuable tool fixing real issue and it's used > by more projects than just kolla-ansible). We could add another script > that would look like extended_start.sh -> source > $CONTAINER_NAME-extended-start.sh and copy all kolla's extended start > scripts to dir with proper naming (I believe this is solution that Sam > came up with shortly after BCN). This is purely techincal and not that > hard to do, much quicker and easier than deprecating API... > > On 5 April 2018 at 12:28, Martin André <m.an...@redhat.com> wrote: > > On Thu, Apr 5, 2018 at 2:16 PM, Paul Bourke <paul.bou...@oracle.com> > wrote: > >> Hi all, > >> > >> This mail is to serve as a follow on to the discussion during > yesterday's > >> team meeting[4], which was regarding the desire to move start scripts > out of > >> the kolla images [0]. There's a few factors at play, and it may well be > best > >> left to discuss in person at the summit in May, but hopefully we can > get at > >> least some of this hashed out before then. > >> > >> I'll start by summarising why I think this is a good idea, and then > attempt > >> to address some of the concerns that have come up since. > >> > >> First off, to be frank, this is effort is driven by wanting to add > support > >> for loci images[1] in kolla-ansible. I think it would be unreasonable > for > >> anyone to argue this is a bad objective to have, loci images have very > >> obvious benefits over what we have in Kolla today. I'm not looking to > drop > >> support for Kolla images at all, I simply want to continue decoupling > things > >> to the point where operators can pick and choose what works best for > them. > >> Stemming from this, I think moving these scripts out of the images > provides > >> a clear benefit to our consumers, both users of kolla and third parties > such > >> as triple-o. Let me explain why. > > > > It's still very obscure to me how removing the scripts from kolla > > images will benefit consumers. If the reason is that you want to > > re-use them in other, non-kolla images, I believe we should package > > the scripts. I've left some comments in your spec review. > > > >> Normally, to run a docker image, a user will do 'docker run > >> helloworld:latest'. In any non trivial application, config needs to be > >> provided. In the vast majority of cases this is either provided via a > bind > >> mount (docker run -v hello.conf:/etc/hello.conf helloworld:latest), or > via > >> environment variables (docker run --env HELLO=paul helloworld:latest). > This > >> is all bog standard stuff, something anyone who's spent an hour learning > >> docker can understand. > >> > >> Now, lets say someone wants to try out OpenStack with Docker, and they > look > >> at Kolla. First off they have to look at something called > set_configs.py[2] > >> - over 400 lines of Python. Next they need to understand what that > script > >> consumes, config.json [3]. The only reference for config.json is the > files > >> that live in kolla-ansible, a mass of jinja and assumptions about how > the > >> service will be run. Next, they need to figure out how to bind mount the > >> config files and config.json into the container in a way that can be > >> consumed by set_configs.py (which by the way, requires the base kolla > image > >> in all cases). This is only for the config. For the service start up > >> command, this need to also be provided in config.json. This command is > then > >> parsed out and written to a location in the image, which is consumed by > a > >> series of start/extend start shell scripts. Kolla is *unique* in this > >> regard, no other project in the container world is interfacing with > images > >> in this way. Being a snowflake in this regard is not a good thing. I'm > still > >> waiting to hear from a real world operator who would prefer to spend > time > >> learning the above to doing: > > > > You're pointing a very real documentation issue. I've mentioned in the > > other kolla thread that I have a stub for the kolla API documentation. > > I'll push a patch for what I have and we can iterate on that. > > > >> docker run -v /etc/keystone:/etc/keystone keystone:latest --entrypoint > >> /usr/bin/keystone [args] > >> > >> This is the Docker API, it's easy to understand and pretty much the > standard > >> at this point. > > > > Sure, using the docker API works for simpler cases, not too > > surprisingly once you start doing more funky things with your > > containers you're quickly reach the docker API limitations. That's > > when the kolla API comes in handy. > > See for example this recent patch > > https://review.openstack.org/#/c/556673/ where we needed to change > > some file permission to the uid/gid of the user inside the container. > > > > The first iteration basically used the docker API and started an > > additional container to fix the permissions: > > > > docker run -v > > /etc/pki/tls/certs/neutron.crt:/etc/pki/tls/certs/neutron.crt:rw \ > > -v /etc/pki/tls/private/neutron.key:/etc/pki/tls/private/ > neutron.key:rw > > \ > > neutron_image \ > > /bin/bash -c 'chown neutron:neutron > > /etc/pki/tls/certs/neutron.crt; chown neutron:neutron > > /etc/pki/tls/private/neutron.key' > > > > You'll agree this is not the most obvious. And it had a nasty side > > effect that is changes the permissions of the files _on the host_. > > While using kolla API we could simply add to our config.json: > > > > - path: /etc/pki/tls/certs/neutron.crt > > owner: neutron:neutron > > - path: /etc/pki/tls/private/neutron.key > > owner: neutron:neutron > > > >> The other argument is that this removes the possibility for immutable > >> infrastructure. The concern is, with the new approach, a rookie operator > >> will modify one of the start scripts - resulting in uncertainty that > what > >> was first deployed matches what is currently running. But with the way > Kolla > >> is now, an operator can still do this! They can restart containers with > a > >> custom entrypoint or additional bind mounts, they can exec in and change > >> config files, etc. etc. Kolla containers have never been immutable and > we're > >> bending over backwards to artificially try and make this the case. We > cant > >> protect a bad or inexperienced operator from shooting themselves in the > >> foot, there are better ways of doing so. If/when Docker or the upstream > >> container world solves this problem, it would then make sense for Kolla > to > >> follow suit. > >> > >> On the face of it, what the spec proposes is a simple change, it should > not > >> radically pull the carpet out under people, or even change the way > >> kolla-ansible works in the near term. If consumers such as tripleo or > other > >> parties feel it would in fact do so please do let me know and we can > discuss > >> and mitigate these problems. > > > > TripleO uses these scripts extensively, we certainly do not want to > > see them go away from kolla images. > > > > Martin > > > >> Cheers, > >> -Paul > >> > >> [0] https://review.openstack.org/#/c/550958/ > >> [1] https://github.com/openstack/loci > >> [2] > >> https://github.com/openstack/kolla/blob/master/docker/base/ > set_configs.py > >> [3] > >> https://github.com/openstack/kolla-ansible/blob/master/ > ansible/roles/keystone/templates/keystone.json.j2 > >> [4] > >> http://eavesdrop.openstack.org/meetings/kolla/2018/kolla. > 2018-04-04-16.00.log.txt > >> > >> ____________________________________________________________ > ______________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: > unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ____________________________________________________________ > ______________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject: > unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev