On 1/13/2017 11:43 AM, Alex Schultz wrote:
Ahoy folks,
So we've been running into issues with the addition of the cell v2
setup as part of the requirements for different parts of nova. It was
recommended that we move the conversation to the ML to get a wider
audience. Basically, cell v2 has been working it's way into a
required thing for various parts of nova for the Ocata cycle. We've
hit several issues[0][1] because of this. I put a wider audience than
just nova because deployment tools need to understand how this works
as it impacts anyone installing or upgrading.
What is not clear is what is the expectation around how to install and
configure cell v2. When we hit the first bug in the upgrade, we
reached out in irc[2] around this and it seemed that there was little
to no documentation around how this stuff all works. There are
mentions of these new commands in the release notes[3] but it's not
clear on the specifics for both the upgrade process and also a fresh
install. We attempted to just run simple_cell_setup in the puppet
(and tripleo downstream) because we assumed this would handle all the
things. It's become clear that this is not the case. The latest
bug[1] has shown that we do not have a proper understanding of what it
means to setup cell v2, so I'd like to use this email to start the
conversation as it impacts anyone either install Ocata fresh or
attempting some sort of Newton -> Ocata upgrade.
Additionally after some conversations today on irc[4], it's also
become clear there is some disconnect around understanding between
nova folks and people who deploy as to how this thing would ideally
work. So, what I would like to know is how should people be
installing and configuring nova cell v2? Specifically what are the
steps so that the deployment tools and operators can properly handle
these complexities. What are the assumptions being baked into
simple_cell_setup? It seems to assume computes should exist before
the cell simple setup where as traditionally computes are the last
thing to be setup (for new installs).
So, help?
Thanks,
-Alex
[0] https://bugs.launchpad.net/tripleo/+bug/1649341
[1] https://bugs.launchpad.net/nova/+bug/1656276
[2]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2016-12-12.log.html#t2016-12-12T17:38:56
[3] http://docs.openstack.org/releasenotes/nova/unreleased.html#id12
[4]
http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-01-13.log.html#t2017-01-13T14:11:37
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Thanks for starting this thread.
First, I want to apologize for the lack of communication and
documentation around this. I know this is frustrating. We've been very
heads down on getting the changes out and getting devstack/grenade
working with this stuff that we haven't taken the time to document it
outside of the release notes, which isn't adequate.
Without going into details, there was a major change in personnel this
release for working on cells v2 so we've been doing some catch-up and
that's part of why things are a bit scatter-brained.
Based on the discussion in IRC this morning, I took some time to try and
capture some of the immediate issues/todos/questions in the cells v2
wiki [1].
Documenting this is going to be a priority. We should have something up
for review in Nova by next week (like Monday), at least a draft.
I think it's also important to realize (for the nova team) that we've
been thinking about cells v2 deployment from an upgrade perspective,
which I think is why we had the simple_cell_setup command asserting that
you needed computes already registered to map to hosts and cells. As
noted, this is not going to work in a fresh install deployment where the
control services are setup before the computes. We're working on
addressing that in [2].
To compare with how simple_cell_setup works, the recently created
'nova-status upgrade check' command [3] is OK with there being no
compute nodes yet (it does fail if you don't have any cell mappings
though). It's OK with that because of the fresh install scenario. It
doesn't fail but it does report that you need to remember to discover
new hosts and map them once they are registered.
So for whatever reason the existing commands and code were written under
the assumption that you'd first create computes (or already have them)
and then map those to a cell, and we need to adjust the tooling for the
scenario that you want to create the cell first and map hosts later. I
think today grenade and devstack are working the same way as far as
setting up cells v2, but eventually I think we're going to want to make
grenade do the more specific upgrade path where we expect a compute node
to already exist and get mapped, and change the devstack path to use the
fresh install scenario where there might not be a compute node when the
cell is created, but then map it later after starting the nova-compute
service. That will build on top of [2].
We've also kicked around the idea of the computes auto-registering with
a cell when they are created, but we don't know which cell (if there are
multiple) they'd register with, and we've wanted to avoid an up-call
from the computes to the API to do that sort of thing. But that would
take away the manual step of running the discover_hosts command to map
the compute node to a cell. The way things are working right now when
getting instances at the API layer is we look for a cell mapping and if
the instance isn't in a cell, we fallback to the main database (no cells
configured). A warning gets logged when that happens. I think we can
live with that for now in Ocata and then when we get into Pike start
considering that an error but talk about auto-registration, i.e. maybe
we auto-register with a single cell and it's only an error if there are
multiple cells and we don't know which to register with. Or we mark a
specific cell as the 'default' or 'staging' cell that all new compute
nodes go into, and then provide a CLI or API to move hosts/instances to
another cell if needed. Anyway, those are longer-term concerns/ideas
right now.
First priority right now is going to be fixing simple_cell_setup [2] and
documentation, including not only the deployment steps, but also
documenting the various commands themselves, including inputs/outputs,
assumptions made and in what situations you can or should run them.
[1] https://wiki.openstack.org/wiki/Nova-Cells-v2
[2] https://review.openstack.org/#/c/420079/
[3]
https://github.com/openstack/nova/blob/31774fa2285a29e7b711a90586d120aaab0b627d/nova/cmd/status.py#L120-L173
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
