Hi all, One of our priority goals for Liberty was the adoption of oslo.versionedobjects in order for Cinder to achieve ability to do rolling upgrades. We weren't successful with that in L, and work got postponed to Mitaka. I want to highlight remaining work in that topic as well as other pieces that are still missing in order for Cinder to support no-downtime-upgrades.
Basically in order to be able to perform such upgrade we need make sure that our services are compatible between versions. There is a set of problems that needs to be solved: * Non-compatible DB migrations (e.g. dropping or altering DB columns). * Non-compatible RPC API changes (e.g. rename of an argument of a RPC method). * Non-compatible changes inside objects/dicts sent over RPC (e.g. removal of a key there). Good explanation of how Nova solves these can be found in a series of posts by Dan Smith - [1][2][3]. I'll walk through all of these. DB migrations ------------- Since Juno no non-compatible DB migration was merged. We may stick to this approach and allow only additive migrations to be performed (we probably may allow dropping columns in further release - require that only two subsequent releases are compatible). This is easy to prevent using an unit test [4]. Another solution would be to implement online schema migrations. This was implemented in Nova [5], but is considered to be unstable and experimental. RPC API compatibility --------------------- We're already versioning our RPC layer, but we aren't actually benefiting from it - we don't support RPC API pinning and don't pay attention to merge only changes that are backward compatible. This requires cultural change in reviewing and I think we should discuss the approach at the Design Summit sprint. Versioned Objects ----------------- Right now there's a few outstanding DB-based objects: * CGSnapshot (in review). * Volume (partly in review). You can find patches in [5]. Apart from that I think we need to convert dictionaries sent over RPC to versioned objects. This would include: * request_spec (scheduler.rpcapi) * filter_properites (scheduler.rpcapi) * capabilities (scheduler.rpcapi) - I'm not sure on this one… Changing this is required for us to be able to remove or rename fields in these dictionaries and still be able to provide interoperability of services working in different versions. I would love to get some feedback on these thoughts and possibly start a pre-summit discussion on the whole topic. [1] http://www.danplanet.com/blog/2015/10/05/upgrades-in-nova-rpc-apis/ [2] http://www.danplanet.com/blog/2015/10/06/upgrades-in-nova-objects/ [3] http://www.danplanet.com/blog/2015/10/07/upgrades-in-nova-database-migrations/ [4] https://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_migrations.py#L186-L227 [5] http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/online-schema-changes.html [6] https://review.openstack.org/#/q/status:open+project:openstack/cinder+branch:master+topic:bp/cinder-objects,n,z __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
