On 5/4/16 12:52 PM, Gregory Haynes wrote:
On Wed, May 4, 2016, at 08:55 AM, Flavio Percoco wrote:
On 04/05/16 15:05 +0000, Amrith Kumar wrote:
I'm emailing the ML on the subject of a review ongoing in the Trove project
regarding image building[1].
TL;DR
One of the most frequent questions that new users of Trove ask is how and where
to get guest images with which to experiment with Trove, and how to build these
images for themselves. While documentation about this exists in multiple places
(including [2], [3]) this is still something that can do with some improvement.
Trove currently uses diskimage-builder for building images used in testing the
product and these can serve as a good basis for anyone wishing to build an
image for their own use of Trove. The review [1] makes the argument for the
libguestfs based approach to building images and advocates that Trove should
use this instead of diskimage-builder.
At the summit we discussed the possibility of providing an implementation
that
would allow for both DIB and libguestfs to be used but to give priority
to DIB.
Since there's no real intention of just switching tools at this point, I
believe
it'd be good to amend the spec so that it doesn't mention libguestfs
should be
used instead of DiB.
The goal at this stage is to provide both and help these move forward.
I believe that a broader discussion of this is required and I appreciate Greg
Haynes' proposal at the design summit to have this discussion on the ML. I took
the action item to bring this discussion to the ML.
Details follow ...
Before going further, I will state my views on these matters.
1. It is important for the Trove project to do things quickly to make it easier
for end users who wish to use Trove and who wish to build their own images. I
am not concerned what tool or tools a person will use to build these images.
++. One of the biggest issues I see users of DIB hit is ease of use for
'just make me an image, I don't care about twiddling knobs'. A wrapper
script in trove is one way to help with this, but I am sure there are
other solutions as well... maybe by rethinking some of our fear about
using elements as entry points to an image build, or by simply making
element's with better defaults.
2. If we provide multiple alternatives to image building as part of the Trove
project, we should make sure that images built with all sets of tools are
equivalent and usable interchangeably. Failing to do this will make it harder
for users to use Trove because we will be providing them with a false choice
(i.e. the alternatives aren't really alternatives). This is harder than it
sounds given the combination of tools, operating systems, and the source(s)
from which you can get database software.
Maintaining both in the long run will be harder especially because, as
you
mentioned, the output must be usable interchangeably. However, I think
we're at
a point, based on the comments in [1] made by Pino Toscano, Luigi Toscano
and
some other folks that it'd be beneficial for us to have this discussion
and to
also experiment/test other options.
The Sahara team seems to be going in a direction that differs with the
one used
by the infra team and the one we're headed to (although they overlap in
some
areas).
I would highly recommend against having two sets of image building code
for Trove - given DIB's current design there should not be any need for
this and there's a HUGE downside to maintaining two sets of code to do
the same thing in-tree. Ideally a single set of code would be used while
being able to be run in different environments if there are mutually
exclusive requirements being proposed by users.
Well, certainly one downside in the case of Trove (and probably
elsewhere) with DIB is the src tree matrix of datastore-by-distro
elements required to support various guest image combinations, leading
to a proliferation of directories and files. We feel this can be greatly
simplified using a libguestfs approach using a minimal set of bash and
directly applicable data files (e.g., systemd unit files, conf files,
etc.).
What seemed very apparent to me in the summit session is that there are
a set of issues for Trove relating to image building, mostly relating to
reliability and ease of use. There was no one who even mentioned let
alone strongly cared about the issues which actually differentiate the
existing DIB build process from libguestfs (which is isolation). If that
has changed for some reason, then my recommendation would be to use a
tool like virt-dib which will allow for a single image building code
base while solving all the raised issues in the spec. I suspect when
this is tried out the downsides to booting a VM will highly outweigh the
benefits for almost all users (especially in trove's gate),
Anecdotally, it takes the same amount of time for a CentOS7 MySQL build
(~ 7 minutes) with either toolchain.
but if the
libguestfs docs are to be believed this should be trivial to try out.
Not quite sure what you mean by "to be believed"?
3. Trove already has elements for all supported databases using DIB in the
trove-integration project but these elements are not packaged for customer use.
Making them usable by customers is a relatively small effort including
providing a wrapper script (derived from redstack[4]) and providing an element
to install the guest agent software from a fixed location in addition to the
development and testing version that is better suited to Trove development [5]
and [6].
4. My comments on various patch sets in the review[1].
I agree with Monty and Greg Haynes that we should understand the deficiencies if any in
DIB, and if it is in fact the case that they are "intractable/unsolvable", we
should switch toolchains. This discussion should include issues faced by the Trove team
as well as other teams that may have faced problems with DIB (such as the sahara team who
described some of them in the past).
++
Agreed with the above. I'm think collaboration should be the preferred
way. I
don't think I've enough technical insight on this topic to provide a
detailed
list of things that are good/bad on either of these tools but I wanted to
mention that I believe providing support for both in the short run is
good for
us and it helps to make a better decision on what tool works best for the
project.
Rewriting image building code in order to find out if we want to use a
tool seems completely backwards. Obviously, if some external team wants
to do this there's nothing stopping them, but what we should focus on
are what problems actually effect out user base and what we can do to
solve them. We should *not* be focusing on finding ways to support
various image building frameworks without a clear benefit to doing so.
The various image building frameworks have been noted here
http://docs.openstack.org/image-guide/create-images-automatically.html
including libguestfs. So it's not like it is an unknown quantity. In the
interest of innovation I'm not sure I understand the hearty reluctance
to explore this path. We are proposing simply another Trove repo with an
alternate (and recognized) image build method. This is not displacing
any established tool for Trove; such a tool doesn't exist today. The
elements in trove-integration don't really count since they are largely
developed for Ubuntu only, inject Trove guestagent src from git only,
and, beyond MySQL 5.6, are not tested by the gate.
There's someone willing to do the job and spend sometime doing the
research.
This same person will provide feedback in addition to the one already
provided
in [1].
Sorry for not providing much technical details now but I did want to
share the
above. Thanks for starting this thread, I believe this discussion in the
ML will
be beneficial.
Flavio
Thanks,
-amrith
[1] https://review.openstack.org/#/c/295274/
[2] http://docs.openstack.org/developer/trove/dev/building_guest_images.html
[3]
https://git.openstack.org/cgit/openstack/diskimage-builder/tree/README.rst#writing-an-element
[4]
http://git.openstack.org/cgit/openstack/trove-integration/tree/scripts/redstack
[5]
http://git.openstack.org/cgit/openstack/trove-integration/tree/scripts/files/trove-guest.systemd.conf
[6]
http://git.openstack.org/cgit/openstack/trove-integration/tree/scripts/files/trove-guest.upstart.conf
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
@flaper87
Flavio Percoco
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
[email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Email had 1 attachment:
+ signature.asc
1k (application/pgp-signature)
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
