On Thursday at the summit the Cinder team had a session about the
proposed volume attach/detach API changes that John Griffith is working
on. The etherpad for that session is here:
https://etherpad.openstack.org/p/ocata-nova-summit-cinder-session
On Friday we had a joint nova/cinder session to go over the proposals
and POCs with the larger teams. The etherpad for that session is here:
https://etherpad.openstack.org/p/ocata-nova-summit-cinder-session
John Griffith has been working on two different versions of adding a new
series of APIs that deal with attachment CRUD operations which nova
would plan on using, and which should also help ease in the support for
volume multiattach.
John's first iteration on this is slightly closer to what we already
have today, but with Cinder storing more information (like the host
connector and connection_info) and providing more information to Nova
like whether or not a connection is shared - which varies based on the
Cinder backend. That version still relies on nova to reserve (lock) the
volume in nova-api before casting to nova-compute to create the
attachment and connect it on the host using os-brick.
John is working on a second POC here:
https://review.openstack.org/#/c/387712/
That is slightly different and might eliminate the need for nova to call
both os-reserve and create_attachment. Ideally nova gets to the point of
simply creating the attachment (which locks the volume) in nova-api,
then in nova-compute nova will update the attachment with the host
connector (from os-brick) and then connect the volume on the host (using
os-brick). If anything fails, nova would delete the attachment record in
Cinder, thus unlocking the volume (like detach, terminate-connection and
unreserve today).
One of the main issues we're still trying to sort out is nova knowing
when it's safe to disconnect an attachment, mainly in the case of shared
storage backends. We could have a race where nova is getting a
connection count from cinder which basically says it's safe to
disconnect from the host because it's the only connection (A), but
meanwhile another request creates an attachment (B) and when nova
disconnects A is also disconnects B. We think that we can work through
this with a lock in nova so that you can't be attaching with a shared
connection on a host at the same time that the shared connection is
being detached from that host.
We also need to sort out the live migration flow where you have a
non-multiattach volume but you're using it to create two connections for
the same instance but on different hosts. There would most likely need
to be some logic in the Cinder API to allow this, since otherwise
normally you can't have more than one attachment to a non-multiattach
volume. Supporting this, though, would make the volume-backed live
migration flow in Nova quite simple, and would be similar to how
Nova/Neutron plan to handle multiple port bindings during live migration.
With respect to upgrades, we haven't gotten too in-depth into that yet.
With the proposed flows Nova would be tracking an attachment_id in the
block_device_mappings for an instance. Nova may or may not be storing
the connection_info anymore since Cinder would store that and Nova can
just get it via the attachment record. But as part of an upgrade we'd
likely need to migrate old existing BDM records to create an attachment
record in Cinder and then store that in the nova BDM. It's TBD on if
something like that could happen with a nova-manage command, or if it
would be more of an online migration when operations are performed on
older volumes.
The nova side flows with all of this are detailed in John Garbutt's nova
spec here:
https://review.openstack.org/#/c/373203/
We're still having weekly nova/cinder cross-project meetings at 1700 UTC
on Mondays:
http://eavesdrop.openstack.org/#Cinder,_Nova
And have started doing those with Google Hangouts to speed along the
discussion. Ildikov has been taking notes during the IRC meeting though
so the minutes/highlights are tracked in case the hangout isn't recorded.
All in all we are making progress, slowly but surely, on the design and
working through some of the edge cases. If we can come to an agreement
on the overall workflow early enough in Ocata then it might be possible
to get the Cinder API changes in and then the Nova side can possibly get
landed in Pike. However, given the Cinder team's focus on technical debt
and stability for Ocata it might be a stretch. Either way, having an
agreed on design and POC code by the end of Ocata would be a major
accomplishment and greatly help getting volume multiattach supported in
Pike.
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
