[openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Hi everyone, The starting point for this post is a specific Neutron sub-project (networking-bgpvpn) but I believe the issues raised are shared with other neutron stadium project and possibly relevant beyond Neutron, to projects tagged release-independent in general. In the context of the networking-bgpvpn project, we have a few unsolved issues related to branches, more specifically about which branches we can create to host active development to make our subprojet (a Neutron service plugin) work with the last Openstack release and to backport it with to earlier releases. Here are precisely the assumptions that we've made, largely based on the fact that the project is tagged 'release-independent' (meaning "completely bypass the 6-month cycle and release independently" [1]): a) that we can make a release targeting Liberty much later than the release date of Liberty b) that we could make releases more frequent than the 6-month cycle ; not only bugfix releases but also feature releases c) that the idea of a release-independent project being backported to work with past Openstack releases is acceptable (of course, without requiring any change in release-managed projects, something largely possible in many cases for projects such as a Neutron service plugin or an ML2 driver) Note that we aren't the only big tent project having this kind of expectations (e.g. [3]). The rest of this email follows from these assumptions. To do (a) and (c) we need separate branches in which to do active work. We have understood clearly that given the contract around 'stable/*' branches, the branches in which to do this active work cannot be named 'stable/*'. Where we are today: - we do active development, planning a liberty release soon, on our 'master' branch - we have a 'backport/kilo' and a 'backport/juno' branches in which we plan to make the necessary changes to adapt with Neutron kilo and juno; these branches were created by Neutron core devs for us [5], based on the common understanding that choosing 'stable/*' branch names for branches with active development would be a huge no no Let me take a few example of the issues we hit... I apologize in advance for imprecisions that may result from my limited experience with the Openstack project at large, and CI subtleties in particular. ### Continuous Integration issue Our master branch is written to work with stable/liberty of other projects we depend on (neutron, python-neutronclient, devstack for the CI devstack VMs). The issue we hit, at least this is my understanding, is that the CI system (at least the devstack VM parts) has a built-in implicit assumption that branch X of project foo is meant to work with branch X of project bar, but that this assumption is not true in our case. The solution we could consider is to tweak CI gate jobs to add jobs that force a specific branch to be used for Openstack key project our job depend on (e.g. devstack, neutron, in our case), and filter which of these jobs would be triggered for each specific branch of our project. For instance: - define a gate-install-dsvm-networking-bgpvpn-kilo with branch-override stable/kilo (will result in creating a devstack stable/kilo with Neutron stable/kilo inside) - configure zuul to apply this job *only if* the gerrit change triggering the test is for our backport/kilo branch (do that again for our juno branch, and again to map our master to Openstack stable/liberty) It seems like it would work, although possible too verbose. With the infra team support we already have something close to this configuration in place for our master (two jobs, one to test with master, one to force testing against stable/liberty). [4] ### Synchronizing requirements issue Trying to be good pupils we had activated the check-requirements jobs in our zuul config, and the requirements proposal bot as well. Just like the CI system, these requirements tools have a built-in implicit assumption that branch X of project foo is meant to work with branch X of project bar, which breaks in our case. The issues we have are the following: 1- the botis proposing on our 'master' branch, updates to requirements.txt to make us in sync with the requirements 'master' branch, which is irrelevant since our master in facts targets Liberty -- this is easily "solved": we just ignore those proposed changes 2- the bot won't be able to propose anything relevant on our backport/kilo and backport/juno branches, since he does not know they target stable/kilo and stable/juno -- we can live with that and update requirements ourselves 3- we can't update our requirements.txt on our master branch: we want to add requirements conforming with Liberty, but the check-requirements jobs thinks this is wrong since it checks our master branch against its master branch (that now has mitaka requirements) 4- same as 3 for our backport/kilo and backport/juno branches (the job does not know t
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Thomas Morin wrote: > The starting point for this post is a specific Neutron sub-project > (networking-bgpvpn) but I believe the issues raised are shared with > other neutron stadium project and possibly relevant beyond Neutron, to > projects tagged release-independent in general. > > In the context of the networking-bgpvpn project, we have a few unsolved > issues related to branches, more specifically about which branches we > can create to host active development to make our subprojet (a Neutron > service plugin) work with the last Openstack release and to backport it > with to earlier releases. > > Here are precisely the assumptions that we've made, largely based on the > fact that the project is tagged 'release-independent' (meaning > "completely bypass the 6-month cycle and release independently" [1]): > a) that we can make a release targeting Liberty much later than the > release date of Liberty > b) that we could make releases more frequent than the 6-month cycle ; > not only bugfix releases but also feature releases > c) that the idea of a release-independent project being backported to > work with past Openstack releases is acceptable (of course, without > requiring any change in release-managed projects, something largely > possible in many cases for projects such as a Neutron service plugin or > an ML2 driver) So we have three models. The release:independent model is for projects that don't follow the common development cycle, and therefore won't make a "liberty" release. The release:cycle-with-milestones model is the traditional "one release at the end of the cycle" model, and the release:cycle-with-intermediary model is an hybrid where you follow the development cycle (and make an end-of-cycle release) but can still make intermediary, featureful releases as necessary. The difference between the two groups is the concept of per-cycle branches (the stable/liberty branch which is used to maintain backports following the stable branch policy). Projects following the release:independent model should not have a stable/liberty branch, since they didn't formally do a liberty release. Those independent projects that have a stable/liberty branch are actually all release:cycle-with-intermediary projects that ignore their true nature. Looking at your specific case, it appears you could adopt the release:cycle-with-intermediary model, since you want to maintain a branch mapped to a given release. The main issue is your (a) point, especially the "much later" point. Liberty is in the past now, so making "liberty" releases now that we are deep in the Mitaka cycle is a bit weird. When are you likely to start shipping your liberty branch ? Maybe we need a new model to care for such downstream projects when they can't release in relative sync with the projects they track. > Note that we aren't the only big tent project having this kind of > expectations (e.g. [3]). > > The rest of this email follows from these assumptions. > > To do (a) and (c) we need separate branches in which to do active work. > We have understood clearly that given the contract around 'stable/*' > branches, the branches in which to do this active work cannot be named > 'stable/*'. > > Where we are today: > - we do active development, planning a liberty release soon, on our > 'master' branch > - we have a 'backport/kilo' and a 'backport/juno' branches in which we > plan to make the necessary changes to adapt with Neutron kilo and juno; > these branches were created by Neutron core devs for us [5], based on > the common understanding that choosing 'stable/*' branch names for > branches with active development would be a huge no no > [...] So we had that discussion at the last design summit: how should we name those branches that track a given release cycle but do not necessarily follow the common stable branch policy. My initial idea was to reserve usage of the stable/* name to things following stable branch policy. But that creates a number of issues on the infra side, some of which you've already hit. So we discussed the idea of calling them all stable/*, and use a tag to designate which of those branches actually follow the standard stable branch policy (rather than relying on the branch name to determine that). > [...] > ### Doing multiple releases inside one 6-month Openstack cycle issue > > Our initial plan was to fork a 'stable/liberty' branch from our master > at the same time as our first release. > But after this we don't know how to work on new features for a release > still targeting Liberty: > - adding features on stable/liberty is a no no > - there is no established practice, as far as we know, to name such > branches, nor to track that they aim to work with Liberty > > We haven't a solution for that right now, and we definitely want guidance. Under the proposed model above, adding "features" to a stable/* branch would be ok. You just would not get the "follows-stable-branch-policy" tag. So I would say the next
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Thanks to Thomas for continuing this discussion about networking-* projects, and to Thierry for his responses below, and Ihar for his earlier guidance. I have a couple further points that I hope may contribute... On 19/11/15 09:46, Thierry Carrez wrote: > Thomas Morin wrote: >> The starting point for this post is a specific Neutron sub-project >> (networking-bgpvpn) but I believe the issues raised are shared with >> other neutron stadium project and possibly relevant beyond Neutron, to >> projects tagged release-independent in general. >> >> In the context of the networking-bgpvpn project, we have a few unsolved >> issues related to branches, more specifically about which branches we >> can create to host active development to make our subprojet (a Neutron >> service plugin) work with the last Openstack release and to backport it >> with to earlier releases. >> >> Here are precisely the assumptions that we've made, largely based on the >> fact that the project is tagged 'release-independent' (meaning >> "completely bypass the 6-month cycle and release independently" [1]): >> a) that we can make a release targeting Liberty much later than the >> release date of Liberty >> b) that we could make releases more frequent than the 6-month cycle ; >> not only bugfix releases but also feature releases >> c) that the idea of a release-independent project being backported to >> work with past Openstack releases is acceptable (of course, without >> requiring any change in release-managed projects, something largely >> possible in many cases for projects such as a Neutron service plugin or >> an ML2 driver) > So we have three models. The release:independent model is for projects > that don't follow the common development cycle, and therefore won't make > a "liberty" release. The release:cycle-with-milestones model is the > traditional "one release at the end of the cycle" model, and the > release:cycle-with-intermediary model is an hybrid where you follow the > development cycle (and make an end-of-cycle release) but can still make > intermediary, featureful releases as necessary. > > The difference between the two groups is the concept of per-cycle > branches (the stable/liberty branch which is used to maintain backports > following the stable branch policy). Projects following the > release:independent model should not have a stable/liberty branch, since > they didn't formally do a liberty release. Those independent projects > that have a stable/liberty branch are actually all > release:cycle-with-intermediary projects that ignore their true nature. > > Looking at your specific case, it appears you could adopt the > release:cycle-with-intermediary model, since you want to maintain a > branch mapped to a given release. The main issue is your (a) point, > especially the "much later" point. Liberty is in the past now, so making > "liberty" releases now that we are deep in the Mitaka cycle is a bit > weird. When are you likely to start shipping your liberty branch ? > > Maybe we need a new model to care for such downstream projects when they > can't release in relative sync with the projects they track. On the other hand this could just be a matter of discipline and planning, and of networking-* projects still being quite new. networking-calico is currently release:independent, and was only created a short time before the Liberty release, and was not completely ready at the time of the Liberty release. But I believe that that it would be achievable if the project targeted moving into sync with the Mitaka cycle, and hence then could be release:cycle-with-milestones or release:cycle-with-intermediary. > >> Note that we aren't the only big tent project having this kind of >> expectations (e.g. [3]). I'd like to say a little more about branch thinking in networking-calico, as I think it may be another useful data point. The last exchange on this, between Ihar and me, was as follows: > > > To get networking-calico into a correct state per the above guideline, I > > > think I'd need/want to > > > > > > - create a stable/liberty branch (from the current master, as there is > > > nothing in master that actually depends on Neutron changes since > > > stable/liberty) > > > > > > - continue developing useful enhancements on the stable/liberty branch - > > > because my primary target for now is the released Liberty - and then > > > merge those to master > > > > Once spinned out, stable branches should receive bug fixes only. No new > > features, db migrations, configuration changes are allowed in stable > > branches. > > > > > - eventually, develop on the master branch also, to take advantage of > > > and keep current with changes in Neutron master. > > > > All new features must go to master only. Your master should always be > > tested and work with neutron master (meaning, your master should target > > Mitaka, not Liberty). > > > > > But is that compatible with the permitted stable branch process? It > > > sounds lik
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Hi Thierry, Thanks for you answers, more below. Thierry Carrez : Thomas Morin wrote: The starting point for this post is a specific Neutron sub-project (networking-bgpvpn) but I believe the issues raised are shared with other neutron stadium project and possibly relevant beyond Neutron, to projects tagged release-independent in general. In the context of the networking-bgpvpn project, we have a few unsolved issues related to branches, more specifically about which branches we can create to host active development to make our subprojet (a Neutron service plugin) work with the last Openstack release and to backport it with to earlier releases. Here are precisely the assumptions that we've made, largely based on the fact that the project is tagged 'release-independent' (meaning "completely bypass the 6-month cycle and release independently" [1]): a) that we can make a release targeting Liberty much later than the release date of Liberty b) that we could make releases more frequent than the 6-month cycle ; not only bugfix releases but also feature releases c) that the idea of a release-independent project being backported to work with past Openstack releases is acceptable (of course, without requiring any change in release-managed projects, something largely possible in many cases for projects such as a Neutron service plugin or an ML2 driver) So we have three models. The release:independent model is for projects that don't follow the common development cycle, and therefore won't make a "liberty" release. The release:cycle-with-milestones model is the traditional "one release at the end of the cycle" model, and the release:cycle-with-intermediary model is an hybrid where you follow the development cycle (and make an end-of-cycle release) but can still make intermediary, featureful releases as necessary. The difference between the two groups is the concept of per-cycle branches (the stable/liberty branch which is used to maintain backports following the stable branch policy). Projects following the release:independent model should not have a stable/liberty branch, since they didn't formally do a liberty release. Those independent projects that have a stable/liberty branch are actually all release:cycle-with-intermediary projects that ignore their true nature. Looking at your specific case, it appears you could adopt the release:cycle-with-intermediary model, since you want to maintain a branch mapped to a given release. I get your point. This would indeed correctly model intermediate releases. But since no other project in Openstack depends on networking-bgpvpn, what is the value of forcing a release of the project synched at the end of a cycle ? (also, there is maybe an implication of release milestones and code freezes that may be overkill for some big tent projects, in particular when they are young) The main issue is your (a) point, especially the "much later" point. Liberty is in the past now, so making "liberty" releases now that we are deep in the Mitaka cycle is a bit weird. "weird" I don't know: isn't it consistent with "release independently" ? When are you likely to start shipping your liberty branch ? We're mostly done and we target November. Maybe we need a new model to care for such downstream projects when they can't release in relative sync with the projects they track. I would think so. The reason is that we want to still do the majority of the work in one branch, to avoid the overhead of cherry-picking between branches a large amount of changes (e.g. if we had been working in a liberty branch since September, all this work would have had to be cherry-picked to our master -- and vice-versa: working in master would have meant cherry-picking everything in the liberty branch). Note that we aren't the only big tent project having this kind of expectations (e.g. [3]). The rest of this email follows from these assumptions. To do (a) and (c) we need separate branches in which to do active work. We have understood clearly that given the contract around 'stable/*' branches, the branches in which to do this active work cannot be named 'stable/*'. Where we are today: - we do active development, planning a liberty release soon, on our 'master' branch - we have a 'backport/kilo' and a 'backport/juno' branches in which we plan to make the necessary changes to adapt with Neutron kilo and juno; these branches were created by Neutron core devs for us [5], based on the common understanding that choosing 'stable/*' branch names for branches with active development would be a huge no no [...] So we had that discussion at the last design summit: how should we name those branches that track a given release cycle but do not necessarily follow the common stable branch policy. My initial idea was to reserve usage of the stable/* name to things following stable branch policy. But that creates a number of issues on the infra side, some of which you've already hit. So we dis
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Thomas Morin wrote: > [...] > But still...one last, very practical, question: until the framework is > adjusted, can we pursue our liberty-targeted work in our master branch, > and our backport work in our backport/kilo and backport/juno branches ? > (of course, we will accept the inconvenience of having these renamed to > stable/x if that becomes the right thing to do) Sure, that shouldn't block you. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
> On Nov 19, 2015, at 4:43 AM, Thierry Carrez wrote: > > Thomas Morin wrote: >> The starting point for this post is a specific Neutron sub-project >> (networking-bgpvpn) but I believe the issues raised are shared with >> other neutron stadium project and possibly relevant beyond Neutron, to >> projects tagged release-independent in general. >> >> In the context of the networking-bgpvpn project, we have a few unsolved >> issues related to branches, more specifically about which branches we >> can create to host active development to make our subprojet (a Neutron >> service plugin) work with the last Openstack release and to backport it >> with to earlier releases. >> >> Here are precisely the assumptions that we've made, largely based on the >> fact that the project is tagged 'release-independent' (meaning >> "completely bypass the 6-month cycle and release independently" [1]): >> a) that we can make a release targeting Liberty much later than the >> release date of Liberty >> b) that we could make releases more frequent than the 6-month cycle ; >> not only bugfix releases but also feature releases >> c) that the idea of a release-independent project being backported to >> work with past Openstack releases is acceptable (of course, without >> requiring any change in release-managed projects, something largely >> possible in many cases for projects such as a Neutron service plugin or >> an ML2 driver) > > So we have three models. The release:independent model is for projects > that don't follow the common development cycle, and therefore won't make > a "liberty" release. The release:cycle-with-milestones model is the > traditional "one release at the end of the cycle" model, and the > release:cycle-with-intermediary model is an hybrid where you follow the > development cycle (and make an end-of-cycle release) but can still make > intermediary, featureful releases as necessary. > > The difference between the two groups is the concept of per-cycle > branches (the stable/liberty branch which is used to maintain backports > following the stable branch policy). Projects following the > release:independent model should not have a stable/liberty branch, since > they didn't formally do a liberty release. Those independent projects > that have a stable/liberty branch are actually all > release:cycle-with-intermediary projects that ignore their true nature. > > Looking at your specific case, it appears you could adopt the > release:cycle-with-intermediary model, since you want to maintain a > branch mapped to a given release. The main issue is your (a) point, > especially the "much later" point. Liberty is in the past now, so making > "liberty" releases now that we are deep in the Mitaka cycle is a bit > weird. When are you likely to start shipping your liberty branch ? > > Maybe we need a new model to care for such downstream projects when they > can't release in relative sync with the projects they track. In my mind the “independent” release model was originally meant to mean that the project was completely on their own, doing potentially incorrect and random releases. It wasn’t something I anticipated projects *wanting* to use. It evolved to mean something closer to the opposite of the “managed” tag, but I think we should pull back from that use. We want projects to clearly indicate which of the other cycle-oriented models they intend to follow, and we want something cycle-based for most projects to help distributors and deployers understand which versions of things should be used together. If neither of the existing cycle-based tags meets the needs of a large number of projects, then we should have a clear description of the model actually being followed so we can tag the projects following it. That may mean, in this case, a cycle-with-intermediary-following or something similar, to mean “we have cyclical releases but they come after the cycle of most of the other projects”. On the other hand, it would be better to work to sync up development. If you’re releasing long after the other projects do, distributors may not prioritize packaging the project at all. > >> Note that we aren't the only big tent project having this kind of >> expectations (e.g. [3]). >> >> The rest of this email follows from these assumptions. >> >> To do (a) and (c) we need separate branches in which to do active work. >> We have understood clearly that given the contract around 'stable/*' >> branches, the branches in which to do this active work cannot be named >> 'stable/*'. >> >> Where we are today: >> - we do active development, planning a liberty release soon, on our >> 'master' branch >> - we have a 'backport/kilo' and a 'backport/juno' branches in which we >> plan to make the necessary changes to adapt with Neutron kilo and juno; >> these branches were created by Neutron core devs for us [5], based on >> the common understanding that choosing 'stable/*' branch names for >> branches with active development
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
> On Nov 19, 2015, at 9:28 AM, Thomas Morin wrote: > > Hi Thierry, > > Thanks for you answers, more below. > > Thierry Carrez : >> Thomas Morin wrote: >>> The starting point for this post is a specific Neutron sub-project >>> (networking-bgpvpn) but I believe the issues raised are shared with >>> other neutron stadium project and possibly relevant beyond Neutron, to >>> projects tagged release-independent in general. >>> >>> In the context of the networking-bgpvpn project, we have a few unsolved >>> issues related to branches, more specifically about which branches we >>> can create to host active development to make our subprojet (a Neutron >>> service plugin) work with the last Openstack release and to backport it >>> with to earlier releases. >>> >>> Here are precisely the assumptions that we've made, largely based on the >>> fact that the project is tagged 'release-independent' (meaning >>> "completely bypass the 6-month cycle and release independently" [1]): >>> a) that we can make a release targeting Liberty much later than the >>> release date of Liberty >>> b) that we could make releases more frequent than the 6-month cycle ; >>> not only bugfix releases but also feature releases >>> c) that the idea of a release-independent project being backported to >>> work with past Openstack releases is acceptable (of course, without >>> requiring any change in release-managed projects, something largely >>> possible in many cases for projects such as a Neutron service plugin or >>> an ML2 driver) >> So we have three models. The release:independent model is for projects >> that don't follow the common development cycle, and therefore won't make >> a "liberty" release. The release:cycle-with-milestones model is the >> traditional "one release at the end of the cycle" model, and the >> release:cycle-with-intermediary model is an hybrid where you follow the >> development cycle (and make an end-of-cycle release) but can still make >> intermediary, featureful releases as necessary. >> >> The difference between the two groups is the concept of per-cycle >> branches (the stable/liberty branch which is used to maintain backports >> following the stable branch policy). Projects following the >> release:independent model should not have a stable/liberty branch, since >> they didn't formally do a liberty release. Those independent projects >> that have a stable/liberty branch are actually all >> release:cycle-with-intermediary projects that ignore their true nature. >> >> Looking at your specific case, it appears you could adopt the >> release:cycle-with-intermediary model, since you want to maintain a >> branch mapped to a given release. > > I get your point. This would indeed correctly model intermediate releases. > But since no other project in Openstack depends on networking-bgpvpn, what is > the value of forcing a release of the project synched at the end of a cycle ? It raises your chances of having your project packaged in distributions along with the other compatible components, and it clearly tells deployers of your project which versions are meant to go with those other components. > > (also, there is maybe an implication of release milestones and code freezes > that may be overkill for some big tent projects, in particular when they are > young) We tend to view the milestone model as suitable for either immature or tightly-coupled projects. Mature, loosely coupled, projects are encouraged to use the intermediary release model. > > >> The main issue is your (a) point, especially the "much later" point. >> Liberty is in the past now, so making >> "liberty" releases now that we are deep in the Mitaka cycle is a bit >> weird. > > "weird" I don't know: isn't it consistent with "release independently” ? Again, it’s about signaling where your project fits into the ecosystem of other projects. > >> When are you likely to start shipping your liberty branch ? > > We're mostly done and we target November. > >> >> Maybe we need a new model to care for such downstream projects when they >> can't release in relative sync with the projects they track. > > I would think so. > > The reason is that we want to still do the majority of the work in one > branch, to avoid the overhead of cherry-picking between branches a large > amount of changes (e.g. if we had been working in a liberty branch since > September, all this work would have had to be cherry-picked to our master -- > and vice-versa: working in master would have meant cherry-picking everything > in the liberty branch). We call them stable branches because we don’t expect them to receive a lot of new features. If you depend on having the final release of neutron for a series available before you can finalize your project, that is indeed a new model. But as I said in my other email, it’s not clear that’s something desirable, and it would be better to have the stadium projects released closer in sync with core neutron. > >
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Thu, Nov 19 2015, Doug Hellmann wrote: > In my mind the “independent” release model was originally meant to mean that > the project was completely on their own, doing potentially incorrect and > random > releases. It wasn’t something I anticipated projects *wanting* to use. It > evolved to mean something closer to the opposite of the “managed” tag, but I > think we should pull back from that use. We want projects to clearly indicate > which of the other cycle-oriented models they intend to follow, and we want > something cycle-based for most projects to help distributors and deployers > understand which versions of things should be used together. > > If neither of the existing cycle-based tags meets the needs of a large number > of projects, then we should have a clear description of the model actually > being followed so we can tag the projects following it. That may mean, in this > case, a cycle-with-intermediary-following or something similar, to mean “we > have cyclical releases but they come after the cycle of most of the other > projects”. Gnocchi is applying "release early, release often" so there is no really any big cycle like older OpenStack projects. Major or minor versions are released from time to time, and more often than 6 months in general. It would be good to support that as being *normal*, not "potentially incorrect and random"! > It’s unfortunate that we have so many tools that depend on the “stable/“ > prefix. I looked, and in addition to devstack-gate the release tools assume > the > name stable. It’s not something we can change this cycle, because of the other > priorities, but it would be useful to think about whether we could treat a > series/ prefix the same way as stable/ in those tools, to have an option for > the branch name. I have no idea how much work that would be. In the mean time, > I agree that using the tag as the true indicator that the stable policy is > being followed is a good compromise. And by the way, it's a shame that the release:has-stable-branches cannot be applied for release:independent. We have stable branches in Gnocchi, we cannot have that tag currently for that only reason. Worse, we often hit issue about assumption made about how projects are released. See my recent thread about the devstack-gate based jobs failing for stable branches. It'd be awesome to free those projects and support more flexible release schedule for project having a different velocity. Cheers, -- Julien Danjou // Free Software hacker // https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Hi Neil, Neil Jerram : [snip] I've since realised that my initial statement above wasn't quite right. In fact, because networking-calico uses Neutron interfaces that are pretty stable (ML2 mech driver, DHCP interface driver, etc.) we have found it manageable until now to develop a single (master) branch of the networking-calico code that works with all of the OpenStack releases (Icehouse..Liberty) that we have tested with; and I'd like if possible to continue doing that. This makes a lot of sense. I believe this can be very relevant for many ML2 mechanism drivers, in particular. It leads me to believe that branch names in a subproject may not be the right way to indicate what Openstack branch, or branch_es_ now that you put this forward, are targeted. Back to my humble suggestion in my previous email... Having an "openstack-target.txt" in a project listing the Openstack branch(es) that this project branch targets could possibly help. On reflection, therefore, I believe it's correct that networking-calico development has been happening, and continues to happen, on its master branch, and I hope that we won't ever need stable branches *for the reason of working with different OpenStack releases* (e.g. if it become too difficult to target many OpenStack releases from a single branch). This scenario would be covered by having multiple branches, each with a different content in "openstack-target.txt". But, well, I don't know if this idea of mine can be relevant. -Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Thu, Nov 19, 2015 at 04:35:24PM +0100, Julien Danjou wrote: > On Thu, Nov 19 2015, Doug Hellmann wrote: > > > In my mind the “independent” release model was originally meant to mean that > > the project was completely on their own, doing potentially incorrect and > > random > > releases. It wasn’t something I anticipated projects *wanting* to use. It > > evolved to mean something closer to the opposite of the “managed” tag, but I > > think we should pull back from that use. We want projects to clearly > > indicate > > which of the other cycle-oriented models they intend to follow, and we want > > something cycle-based for most projects to help distributors and deployers > > understand which versions of things should be used together. > > > > If neither of the existing cycle-based tags meets the needs of a large > > number > > of projects, then we should have a clear description of the model actually > > being followed so we can tag the projects following it. That may mean, in > > this > > case, a cycle-with-intermediary-following or something similar, to mean “we > > have cyclical releases but they come after the cycle of most of the other > > projects”. > > Gnocchi is applying "release early, release often" so there is no really > any big cycle like older OpenStack projects. Major or minor versions are > released from time to time, and more often than 6 months in general. > > It would be good to support that as being *normal*, not "potentially > incorrect and random"! > > > It’s unfortunate that we have so many tools that depend on the “stable/“ > > prefix. I looked, and in addition to devstack-gate the release tools assume > > the > > name stable. It’s not something we can change this cycle, because of the > > other > > priorities, but it would be useful to think about whether we could treat a > > series/ prefix the same way as stable/ in those tools, to have an option for > > the branch name. I have no idea how much work that would be. In the mean > > time, > > I agree that using the tag as the true indicator that the stable policy is > > being followed is a good compromise. > > And by the way, it's a shame that the release:has-stable-branches cannot > be applied for release:independent. We have stable branches in Gnocchi, > we cannot have that tag currently for that only reason. Worse, we often > hit issue about assumption made about how projects are released. See my > recent thread about the devstack-gate based jobs failing for stable > branches. > > It'd be awesome to free those projects and support more flexible release > schedule for project having a different velocity. This is actually the idea of the cycle-with-intermediary model. Projects following this model may release as early and often as they like, and choose a release near the same time as the coordinated release, to be the "cycle" release and the base for the stable branch. // jim __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Thu, Nov 19 2015, Jim Rollenhagen wrote: > This is actually the idea of the cycle-with-intermediary model. Projects > following this model may release as early and often as they like, and > choose a release near the same time as the coordinated release, to be > the "cycle" release and the base for the stable branch. Which is not what Gnocchi does. There's no (strict) following of the cycles of older OpenStack projects, and there are more stable branches than one every 6 months. This tag forces to have e.g. stable/liberty and prohibits any other stable/*. -- Julien Danjou ;; Free Software hacker ;; https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Doug : [snip] I get your point. This would indeed correctly model intermediate releases. But since no other project in Openstack depends on networking-bgpvpn, what is the value of forcing a release of the project synched at the end of a cycle ? It raises your chances of having your project packaged in distributions along with the other compatible components, and it clearly tells deployers of your project which versions are meant to go with those other components. Fair enough. The implicit I had in mind is the relative youth of networking-bgpvpn. What you describe indeed make sense for a mature project. Maybe what is needed is a way to accommodate with the needs of projects in their early days, before they evolve into a cycle release model. The main issue is your (a) point, especially the "much later" point. Liberty is in the past now, so making "liberty" releases now that we are deep in the Mitaka cycle is a bit weird. "weird" I don't know: isn't it consistent with "release independently” ? Again, it’s about signaling where your project fits into the ecosystem of other projects. See above. I think some project aren't mature enough to advertise that they fit in the 6-month cycle release. When are you likely to start shipping your liberty branch ? We're mostly done and we target November. Maybe we need a new model to care for such downstream projects when they can't release in relative sync with the projects they track. I would think so. The reason is that we want to still do the majority of the work in one branch, to avoid the overhead of cherry-picking between branches a large amount of changes (e.g. if we had been working in a liberty branch since September, all this work would have had to be cherry-picked to our master -- and vice-versa: working in master would have meant cherry-picking everything in the liberty branch). We call them stable branches because we don’t expect them to receive a lot of new features. If you depend on having the final release of neutron for a series available before you can finalize your project, that is indeed a new model. This is not really the case we are in. We could technically have targeted to do a liberty release at the same time as Openstack, it happens that the project hadn't yet ramped up quick enough to allow that (project created early June, many discussion on the API in July/August etc.). So we had to accept doing a delayed release, and it means doing active work in a master that targets Liberty. But as I said in my other email, it’s not clear that’s something desirable, and it would be better to have the stadium projects released closer in sync with core neutron. Yes, a key part of the discussion relates to whether there it is desirable or not to allow a project 'master' to target Openstack stable/x. I will would that it does for projects that start, because it will help big tent projects do ramp up at their own pace. Maybe the community will decide that this is not ok for x=too-old-a-release, but I would tend to think that this should be advertized/enforced by tagging rather than tools built-in rules. -Thomas _ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Thu, 19 Nov 2015, Julien Danjou wrote: It would be good to support that as being *normal*, not "potentially incorrect and random"! Yes. The underlying issue in this thread is the dominance of the six month cycle and the way this is perceived to be (any may actually be) a benefit for distributors, marketers, etc. That dominance drives the technological and social context of OpenStack. No surprise that it is present in our tooling and our schedules but sometimes I think it would be great if we could fight the power, shift the paradigm, break the chains. But that's crazy talk, isn't it? However it is pretty clear the dominance is not aligned with at least some of the goals of a big tent. One goal, in particular, is making OpenStack stuff useful and accessible to people or groups outside of OpenStack where release-often is awesome and the needs of the packagers aren't really that important. I reckon (and this may be an emerging consensus somewhere in this thread) we need to make it easier (by declaration) in the tooling to test against whatever is desired. Can we enumerate the changes required to make that go? -- Chris Dent (╯°□°)╯︵┻━┻http://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Julien Danjou wrote: > On Thu, Nov 19 2015, Doug Hellmann wrote: > >> In my mind the “independent” release model was originally meant to mean that >> the project was completely on their own, doing potentially incorrect and >> random >> releases. It wasn’t something I anticipated projects *wanting* to use. It >> evolved to mean something closer to the opposite of the “managed” tag, but I >> think we should pull back from that use. We want projects to clearly indicate >> which of the other cycle-oriented models they intend to follow, and we want >> something cycle-based for most projects to help distributors and deployers >> understand which versions of things should be used together. >> >> If neither of the existing cycle-based tags meets the needs of a large number >> of projects, then we should have a clear description of the model actually >> being followed so we can tag the projects following it. That may mean, in >> this >> case, a cycle-with-intermediary-following or something similar, to mean “we >> have cyclical releases but they come after the cycle of most of the other >> projects”. > > Gnocchi is applying "release early, release often" so there is no really > any big cycle like older OpenStack projects. Major or minor versions are > released from time to time, and more often than 6 months in general. > > It would be good to support that as being *normal*, not "potentially > incorrect and random"! It is now "normal" since it's a supported model. The issue in this thread is more about "independent" projects which actually are following the common cycles and therefore want to track them. I.e. projects that picked "independent" not because they really are independent but because they live in a grey area. > [...] > And by the way, it's a shame that the release:has-stable-branches cannot > be applied for release:independent. We have stable branches in Gnocchi, > we cannot have that tag currently for that only reason. Worse, we often > hit issue about assumption made about how projects are released. See my > recent thread about the devstack-gate based jobs failing for stable > branches. > > It'd be awesome to free those projects and support more flexible release > schedule for project having a different velocity. As I said elsewhere, the current "has-stable-branches" tag is useless and needs to be replaced. Like you said, projects following an alternate release cycle should be able to have "stable branches" (they just won't be "stable/liberty" but something like "stable/2.0". What the tag should be describing is if the stable branches follow the common stable policy, or if they have their own rules. So we plan to discontinue the "has-stable-branches" tag (which currently is mostly a natural consequence of following a release:cycle-with* model) and replace it with a "follows-stable-policy" tag that the stable maint team would grant to compliant projects. -- Thierry Carrez (ttx) signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On 19 November 2015 at 09:43, Thierry Carrez wrote: > > So we have three models. The release:independent model is for projects > that don't follow the common development cycle, and therefore won't make > a "liberty" release. The release:cycle-with-milestones model is the > traditional "one release at the end of the cycle" model, and the > release:cycle-with-intermediary model is an hybrid where you follow the > development cycle (and make an end-of-cycle release) but can still make > intermediary, featureful releases as necessary. > Hmm, then it seems to me that OpenStack-Ansible should be tagged 'release:cycle-with-intermediary' instead of 'release:independent' - is that correct? > Looking at your specific case, it appears you could adopt the > release:cycle-with-intermediary model, since you want to maintain a > branch mapped to a given release. The main issue is your (a) point, > especially the "much later" point. Liberty is in the past now, so making > "liberty" releases now that we are deep in the Mitaka cycle is a bit > weird. > The deployment projects, and probably packaging projects too, are faced with the same issue. There's no guarantee that their x release will be done on the same day as the OpenStack services release their x branches as the deployment projects still need some time to verify stability and functionality once the services are finalised. While it could be easily said that we simply create the branch, then backport any fixes, this is not necessarily ideal as it creates an additional review burden and doesn't really match how the stable branches are meant to operate according to the policy. > Maybe we need a new model to care for such downstream projects when they > can't release in relative sync with the projects they track. > Perhaps. Or perhaps the rules can be relaxed for a specific profile of projects (non core?). __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Thu, Nov 19, 2015, at 12:55 PM, Chris Dent wrote: > On Thu, 19 Nov 2015, Julien Danjou wrote: > > > It would be good to support that as being *normal*, not "potentially > > incorrect and random"! > > Yes. > > The underlying issue in this thread is the dominance of the six month > cycle and the way this is perceived to be (any may actually be) a > benefit for distributors, marketers, etc. That dominance drives the > technological and social context of OpenStack. No surprise that it is > present in our tooling and our schedules but sometimes I think it > would be great if we could fight the power, shift the paradigm, break > the chains. > > But that's crazy talk, isn't it? > > However it is pretty clear the dominance is not aligned with at least > some of the goals of a big tent. One goal, in particular, is making > OpenStack stuff useful and accessible to people or groups outside of > OpenStack where release-often is awesome and the needs of the packagers > aren't really that important. > > I reckon (and this may be an emerging consensus somewhere in this > thread) we need to make it easier (by declaration) in the tooling > to test against whatever is desired. Can we enumerate the changes > required to make that go? "Test whatever is desired" is far to nebulous. We need an actual set of concrete needs and requirements and once you have that you can worry about enumerating changes. I am not sure I have seen anything like this in the thread so far. If you have a stable/X.Y branch or stable/foo but are still wanting to map onto the 6 month release cycle (we know this because you are running devstack-gate) how do we make that mapping? is it arbitrary? is there some deterministic method? Things like this affect the changes necessary to the tools but should be listed upfront. Once we have enumerated the problem we can enumerate the changes to fix it. Clark __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Fri, Nov 20 2015, Clark Boylan wrote: > If you have a stable/X.Y branch or stable/foo but are still wanting to > map onto the 6 month release cycle (we know this because you are running > devstack-gate) how do we make that mapping? is it arbitrary? is there > some deterministic method? Things like this affect the changes necessary > to the tools but should be listed upfront. Honestly, we don't use devstack-gate because we map onto a 6 months release. We use devstack-gate because that seems to be the canonical way of using devstack in the gate. :) Right now, I think the problem I stated in: [openstack-dev] [infra][devstack][gnocchi] Unable to run devstack-gate with stable/1.3 http://lists.openstack.org/pipermail/openstack-dev/2015-November/079849.html is pretty clear. Or if it's not feel free to reply to it and I'll give more information. :) -- Julien Danjou # Free Software hacker # https://julien.danjou.info signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
On Fri, Nov 20, 2015, at 04:32 AM, Julien Danjou wrote: > On Fri, Nov 20 2015, Clark Boylan wrote: > > > If you have a stable/X.Y branch or stable/foo but are still wanting to > > map onto the 6 month release cycle (we know this because you are running > > devstack-gate) how do we make that mapping? is it arbitrary? is there > > some deterministic method? Things like this affect the changes necessary > > to the tools but should be listed upfront. > > Honestly, we don't use devstack-gate because we map onto a 6 months > release. We use devstack-gate because that seems to be the canonical way > of using devstack in the gate. :) Mayne I should've said "because you are running devstack via devstack-gate". Running devstack requires making choices of what services to run based on the 6 month release cycle, > > Right now, I think the problem I stated in: > [openstack-dev] [infra][devstack][gnocchi] Unable to run devstack-gate > with stable/1.3 > http://lists.openstack.org/pipermail/openstack-dev/2015-November/079849.html > > is pretty clear. Or if it's not feel free to reply to it and I'll give > more information. :) Will look and followup there. Clark __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Jesse Pretorius wrote: > [...] > The deployment projects, and probably packaging projects too, are faced > with the same issue. There's no guarantee that their x release will be > done on the same day as the OpenStack services release their x branches > as the deployment projects still need some time to verify stability and > functionality once the services are finalised. The question then becomes: are you making an "x release", or are you making a release "supporting/compatible with the x release". What you are saying is that you need some time because you are downstream, reacting to the x release. That is a fair request: you're actually making a release that supports the x release, you're not in the x release. The line in the sand is based on the date: if you release within the development cycle constraints then you're part of the release, if you release after you're downstream of it, reacting to it. What you need to be able to do in all cases is creating a stable branch to maintain that release over the long run. But what you may not be able to do is to be considered "part of the x release" if you release months after the x release is done and shipped. I'll elaborate on that with a more complete proposal on Monday. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][infra][neutron] branches for release-independent projects targeting Openstack release X
Excerpts from Chris Dent's message of 2015-11-19 20:55:15 +: > On Thu, 19 Nov 2015, Julien Danjou wrote: > > > It would be good to support that as being *normal*, not "potentially > > incorrect and random"! > > Yes. > > The underlying issue in this thread is the dominance of the six month > cycle and the way this is perceived to be (any may actually be) a > benefit for distributors, marketers, etc. That dominance drives the > technological and social context of OpenStack. No surprise that it is > present in our tooling and our schedules but sometimes I think it > would be great if we could fight the power, shift the paradigm, break > the chains. > > But that's crazy talk, isn't it? The cycle pattern affects this discussion most directly because of the distributors, but the social aspects of having the community in a rhythm are valuable to us, and not just consumers of our work. If we're going to have a thing called the "Mitaka Release of OpenStack" we should do that on a schedule. As Thierry says elsewhere in this thread, if a project isn't releasing at the time of the Mitaka release but has something compatible with the Mitaka release, that's still a good thing. It's just not part of the Mitaka release. > However it is pretty clear the dominance is not aligned with at least > some of the goals of a big tent. One goal, in particular, is making > OpenStack stuff useful and accessible to people or groups outside of > OpenStack where release-often is awesome and the needs of the packagers > aren't really that important. Hence the cycle-with-intermediary model. > I reckon (and this may be an emerging consensus somewhere in this > thread) we need to make it easier (by declaration) in the tooling > to test against whatever is desired. Can we enumerate the changes > required to make that go? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
