Thomas, you've made a lot of great points here about the challenges
faced by packaging and packagers of OpenStack. They are valid,
important and need to be considered. I don't want to discount the
work you do because it is awesome, important and I'm well aware that
it can be a huge pain in the ass.
We also, however, need to consider what the future might look like and
at least for some people and situations, the future does not involve
debs or rpms of OpenStack: packages and distributions are more trouble
than they are worth when you want to be managing your infrastructure
across multiple, isolated environments. In that case you want
puppet, ansible, chef, docker, k8s, etc.
Sure all those things _can_ use packages to do their install but for
at least _some_ people that's redundant: deploy from a tag, branch,
commit or head of a repo.
That's for _some_ people. For other people packages are great.
Keep in mind that I'm presenting my own opinion here. I'm quite sure
it is different from, for example, Doug's. It's easy to conflate
arguments such that they appear the same. My position is radical (in
the sense that it is seeking to resolve root causes and institute
fundamental change), I suspect Doug's is considerably more moderate
and evolutionary.
More below.
On Tue, 19 Apr 2016, Thomas Goirand wrote:
On 04/18/2016 02:22 PM, Chris Dent wrote:
targeting an
all-in-one install seems only to serve the purposes of all-in-one rpm-
or deb-based installations.
Though that's what most everyone consumes. Or are you proposing that we
completely stop publishing OpenStack in distros?
I'm not really making a proposal. I'm presenting my thoughts on the
matter to participate, by proxy, in the conversations that will
happen at summit. I'm hoping that a proposal that makes everyone
happy will come out of that. It just so happens that my position on
the matter is probably at one of the extremes. Do I expect it all to
happen like I want? No. Do I hope that my concerns will be
integrated in the discussion? Yes.
Remember that in distros, there's only a single version of a library at
any given time, at the exception of transitions (yes, in Red Hat it's
technically possible to install multiple versions, but the policy
strongly advocates against this).
Yes, which is part of why distros and packaging are limiting in the
cloud environment. When there is a security issue or other bug we don't
want to update those libraries via packages, nor update those libraries
across a bunch of different containers or what not.
We simply want to destroy the deployment and redeploy. With new
stuff. We want to do that without co-dependencies.
In other words, packaging of OpenStack into rpms and debs is a short
branch on the tree of life.
Also, all-in-one is what I use in Debian in my CI, to make sure that
everyone works together, with whatever is uploaded (see the
openstack-deploy package in Debian).
Exactly, because this is what is needed to confirm one of your
requirements (that everything works together). Not everyone has that
requirement.
Many (most?) people won't be doing those kinds of installations.
Most users are consuming packages from distributions. Also, if you're
using containers, probably you will also prefer using these packages to
build your containers: that's the most easy, safe and fast way to build
your containers.
I predict that that is not going to last.
If all-in-one installations are important to the rpm- and deb- based
distributions then _they_ should be resolving the dependency issues
local to their own infrastructure
Who is "they"? Well, approximately 1 person full time for Debian, 1 for
Ubuntu if you combine Corey and David (and Debian and Ubuntu
dependencies are worked on together in Debian), so that's 2 full time
for Ubuntu/Debian. Maybe 2 and a half for RDO if what Haikel told me is
still accurate.
Yes, this is a significant issue and I think is one of the very
complicated aspects of the strange economics of OpenStack. It's been
clear from the start that the amount of people involved at the
distro level has been far too low to satisfy the requirements of the
users of those distributions. Far.
However, that problem is, I think, orthogonal to the question of
effectively creating OpenStack at the upstream (pre-packaging) level.
So "we" wont handle it, even if "we" care, because we're already
understaffed.
I agree. That's a problem for employers of the packagers to solve,
not OpenStack at large. You could make a pretty strong argument, of
course, that OpenStack at large _is_ those employers. That's
unfortunate and isn't the reality we should be aspiring to.
I think making these changes will help to improve and strengthen the
boundaries and contracts between services. If not technically then
at least socially, in the sense that the negotiations that people
make to get things to work are about what actually matters in their
services, not unwinding python dependencies and the like.
Yes, it simplifies the life of developers. But it's going to be hell on
earth for production if we discover a serious vulnerability in a
component, which will be deployed N times, with N versions. Are we then
going to provide N patches? The vulnerability management team is also
understaffed...
See above to see my thinking on this.
One challenge I've run into the past is when devstack
plugin A has made an assumption about having access to a python
script provided by devstack plugin B, but it's not on $PATH or its
dependencies are not in the site-packages visible to the current
context. The solution here is to use full paths _into_ virtenvs.
Please take a step back. Devstack and virtualenv are for development,
not for production. That's not, and should not become, our target.
Applying the reasoning you have there will *not* work for distributions.
Hopefully, that's what Doug is proposing, as he would still like things
to work for distros (right?).
Yes, devstack is for development, but the problem that we're trying
to solve for here is that OpenStack _development_ is held back by
attempting to manage global requirements in the way that we do.
One of the biggest reasons we do that is co-installability. One of
the biggest reasons we target co-installability is for the sake of
the distros.
The problem with all that is that co-installation is _bad_. Don't do
that! If we recognize that a lot of these problems go away.
2/ get a lot of dependencies out. Yes, let's really do that! We're
constantly adding more, I've seen only very few going away. Currently
the global-requirements.txt for stable/mitaka is 377 lines long. If you
grep out "client", "oslo" and "os-", "openstack", comments, empty lines,
and a few others, there's about 260 dependencies left. That's insane at
multiple levels. When are we going to see a movement to clean this mess,
and define real OpenStack standards?
That list looks long because it is used for many many projects that
strive for co-installability. If each project has its own list, the
lists are, individually, much shorter.
If we want there to be a big-tent (a debate worth having) then we're
going to have loads of projects, loads of requirements and a
distinct lack of standards. We're going to have to evolve to cope
with that.
--
Chris Dent (�s°□°)�s�喋擤ォ� http://anticdent.org/
freenode: cdent tw: @anticdent
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
