Hi, There has recently been some speculation around the V3 API and whether we should go forward with it or instead backport many of the changes to the V2 API. I believe that the core of the concern is the extra maintenance and test burden that supporting two APIs means and the length of time before we are able to deprecate the V2 API and return to maintaining only one (well two including EC2) API again.
This email is rather long so here's the TL;DR version: - We want to make backwards incompatible changes to the API and whether we do it in-place with V2 or by releasing V3 we'll have some form of dual API support burden. - Not making backwards incompatible changes means: - retaining an inconsistent API - not being able to fix numerous input validation issues - have to forever proxy for glance/cinder/neutron with all the problems that entails. - Backporting V3 infrastructure changes to V2 would be a considerable amount of programmer/review time - The V3 API as-is has: - lower maintenance - is easier to understand and use (consistent). - Much better input validation which is baked-in (json-schema) rather than ad-hoc and incomplete. - Whilst we have existing users of the API we also have a lot more users in the future. It would be much better to allow them to use the API we want to get to as soon as possible, rather than trying to evolve the V2 API and forcing them along the transition that they could otherwise avoid. - We already have feature parity for the V3 API (nova-network being the exception due to the very recent unfreezing of it), novaclient support, and a reasonable transition path for V2 users. - Proposed way forward: - Release the V3 API in Juno with nova-network and tasks support - Feature freeze the V2 API when the V3 API is released - Set the timeline for deprecation of V2 so users have a lot of warning - Fallback for those who really don't want to move after deprecation is an API service which translates between V2 and V3 requests, but removes the dual API support burden from Nova. End TL;DR. Although its late in the development cycle I think its important to discuss this now rather than wait until the summit because if we go ahead with the V3 API there is exploratory work around nova-network that we would like to do before summit and it also affects how we look at significant effort applying changes to the V2 API now. I'd also prefer to hit the ground running with work we know we need to do in Juno as soon as it opens rather than wait until the summit has finished. Firstly I'd like to step back a bit and ask the question whether we ever want to fix up the various problems with the V2 API that involve backwards incompatible changes. These range from inconsistent naming through the urls and data expected and returned, to poor and inconsistent input validation and removal of all the proxying Nova does to cinder, glance and neutron. I believe the answer to this is yes - inconsistencies in the API make it harder to use (eg do I have a instance or a server, and a project or a tenant just to name a couple) and more error prone and proxying has caused several painful to fix issues for us. So at some point existing users of the API will need to make changes or we'll effectively have to carry two APIs, whether it be inside the V2 API code or split between a V2 implementation and V3 implementation. I think the number of changes required often makes it easier to maintain such a split in different files rather than a single one to avoid large slabs of if/else which makes the underlying code harder to understand and more error prone in the eventual removal. Its also more difficult to use decorators like we have for input validation in V3 if the function also has to support the old behaviour. One approach that has been suggested for retaining the V2 API is to gradually over time mark individual interfaces deprecated, support the new behaviour in parallel and then after the deprecation period remove the original behaviour. With this I think we have to consider that if Open Stack continues to be successful then although we already have an existing user base of the V2 API, that with every release we have more and more new users coming in. In fact in a few cycles we may have more post-Icehouse users than pre-Icehouse ones. But by taking this gradual approach we are basically saying to new users of the API that although we know where we are headed with the API that they can't actually write against it yet and instead have to use the V2 API. And then every cycle they will need to update their apps as we slowly deprecate parts and replace them with the new interface. This seems to be a rather hostile approach to people considering using our API. Also every release we delay releasing the V3 API or defer making backwards incompatible changes to V2 (if that's the route we take), the more users we put into this situation of having to rework the software they use to continue to access our API in the future. Another side effect of this is new features such as tasks (which was one of the significant reasons for not releasing the V3 API in Icehouse) would not be able to be designed the way we want it in the V2 API because it requires changes to the core API. So at least in the short term we'd end up with a suboptimal API for tasks. And then go through the pain of moving to the API we really want. One thing to note is that the transition from V2 to V3 for users of the API it does not have to be a big-bang thing. For example, an application can quite legitimately create a server using the V2 API, attach a volume using the V3 API, detach it using the V2 API and delete the server using the V3 API. It is just a fairly thin layer on top of the rest of Nova. So existing users of the API can decide whether they want to tackle the job of moving from the V2 to V3 API in one big step or in smaller ones over time. We have also done a considerable amount of work in the V3 API in terms of infrastructure which is not always visible to the API user. Such as an improved plugin structure, better isolation between plugins, versioning, better error handling, better input validation etc which would all have to be backported to the V2 API. All in all I think if you compare V2 and V3 API code, the latter is a lot cleaner and easier to maintain. This is a non trivial amount of work that would take both a lot of programmer and reviewer time in Juno, and perhaps overflow into Koala. Time that has already been spent on the V3 API. What I think our plan for the API transition should be is: - Release the V3 API in Juno (it is probably too late to release it in Icehouse at this stage now without a bunch of FFEs, but theoretically we could make the skeleton of the tasks API changes in IceHouse without functional tasks which would allow tasks to be added in a backwards compatible manner in Juno). The Juno release would have both full task and nova-network support. - Feature freeze the V2 API development in Juno or at the very least when the V3 API is marked as current/supported (bug fixes are of course ok). I have several reasons for wanting to do this. The first is to avoid the burden of actively developing two APIs. We already have feature parity in V3 with V2 - with the exception of nova-network which was deliberately left out of the V3 API because it was considered deprecated, but has recently been re-opened for development. So an exception for V2 API nova network development would seem reasonable until the V3 API supports it fully. I think it's a pretty unusual situation where a project decides to continue significant feature development on both the latest version and the next most recent version. We don't for example allow feature development in Havana. And I don't think the situation needs to be any different for V2 once V3 is available. We already have feature parity and we have a reasonable transition plan for existing users so I think its quite reasonable to say to users that if they want new features that they need to access them via the new API. Although as mentioned above, its possible to use the V3 API for new features even if a user is using the V2 API for everything else. So any new features could still be accessed by people who just to only modify their existing V2 API based programs to take advantage of the new functionality without modifying the rest. Also deployers, if they wished, could through policy only expose the parts of the V3 API that they want (even though the core would have to be loaded it doesn't necessarily have to be accessible to everyone). - At summit or the Juno midcycle meetup decide on a release when we will remove the V2 API so existing users of the V2 API have plenty of warning. They can decide either to do a gradual transition or a big-bang move to the V3 API. Ultimately where we set the date for the removal of the V2 API is going to be a balance between needs of users and what we can afford to spend on maintenance of it. But whether we try to evolve the V2 API or move to the V3 API we still need to make that decision. If we choose the V3 API we can also still choose to gradually deprecate the V2 over time (eg remove support for rarely used extensions earlier and point users to the V3 API for that functionality). - For those who *really* *really* don't want to move off the V2 API when the V2 API is removed, there is the possibility of writing a separate translation service which takes V2 API REST requests and translates them to V3 API requests and also does the proxying that the V2 API currently does for glance/neutron/cinder. Its not a trivial job, but should be possible and does remove the burden of supporting two APIs in the Nova tree. In summary, we need to fix the problems with the V2 API which requires changes which are backwards incompatible. I believe releasing the V3 API in Juno and setting a deprecation timeframe for V2 well in advance is the best overall solution we have for both our users (both current and future) and us as developers - it balances the needs of users of our API and what the easiest path for us as developers is. Attempting to "evolve" the V2 API is on the surface attractive because we don't in the short term need to make any decisions around deprecation but will in the longer term involve more pain for everyone. Chris _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev