Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Kuvaja, Erno]

> -Original Message-
> From: Flavio Percoco [mailto:fla...@redhat.com]
> Sent: Monday, January 25, 2016 3:07 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on 
> the
> idea to move it forward
> 
> On 20/01/16 13:23 -0430, Flavio Percoco wrote:
> >Thoughts? Feedback?
> 
> Hey Folks,
> 
> Thanks a lot for the feedback. Great comments and proposals in the many
> replies.
> I've gone through the whole thread and collected the most common
> feedback.
> Here's the summary:
> 
> - The general idea of planning some sort of stabilization for a project is 
> good
>   but considering a cycle for it is terrible. It'd be easier if development
>   cycles would be shorter but the 6-month based development cycles don't
> allow
>   for planning this properly.
> 
> - Therefore, milestones are more likely to be good for this but there has to
> be
>   a good plan. What will happen with on-going features? How does a project
>   decide what to merge or not? Is it really going to help with reviews/bugs
>   backlog? or would this just increase the bakclog?
> 
> - We shouldn't need any governance resolution/reference for this. Perhaps a
>   chapter/section on the project team guide should do it.
> 
> - As other changes in the commuity, it'd be awesome to get feedback from a
>   project doing this before we start encouraging other projects to do the
> same.
> 
> 
> I'll work on adding something to the project team guide that covers the
> above points.
> 
> did I miss something? Anything else that we should add and or consider?
> 

Sorry for jumping the gun this late, but I have been thinking about this since 
your first e-mail and one thing bothers me. Don't we have stabilization cycle 
for each release starting right from the release?

In my understanding this is exactly what the Stable releases Support Phase I is 
accepting bug fixes but no new features. After 6 months the release is moved to 
Phase II where only critical and security fixes are accepted; I think this is 
good example of stabilization cycle and the output is considered solid.

All concerns looked at I think the big problem really is to get the people 
working on these cycles. Perhaps we should encourage more active maintenance on 
our stable branches and see then what we can bring from that to our development 
branch expertise and knowledge wise.

While I'm not huge believer of constant agile development, this is one of those 
things that needs to be lived with and I think stable branches are our best bet 
for stabilization work (specifically when that work needs to land to master 
first). For long term refactoring I'd like to see us using more feature 
branches so we can keep doing the work without releasing it before it's done.

My 2 Euro cents,
Erno

> Cheers,
> Flavio
> 
> --
> @flaper87
> Flavio Percoco
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 27/01/16 11:00 +, Kuvaja, Erno wrote:

-Original Message-
From: Flavio Percoco [mailto:fla...@redhat.com]
Sent: Monday, January 25, 2016 3:07 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the
idea to move it forward

On 20/01/16 13:23 -0430, Flavio Percoco wrote:
>Thoughts? Feedback?

Hey Folks,

Thanks a lot for the feedback. Great comments and proposals in the many
replies.
I've gone through the whole thread and collected the most common
feedback.
Here's the summary:

- The general idea of planning some sort of stabilization for a project is good
  but considering a cycle for it is terrible. It'd be easier if development
  cycles would be shorter but the 6-month based development cycles don't
allow
  for planning this properly.

- Therefore, milestones are more likely to be good for this but there has to
be
  a good plan. What will happen with on-going features? How does a project
  decide what to merge or not? Is it really going to help with reviews/bugs
  backlog? or would this just increase the bakclog?

- We shouldn't need any governance resolution/reference for this. Perhaps a
  chapter/section on the project team guide should do it.

- As other changes in the commuity, it'd be awesome to get feedback from a
  project doing this before we start encouraging other projects to do the
same.


I'll work on adding something to the project team guide that covers the
above points.

did I miss something? Anything else that we should add and or consider?



Sorry for jumping the gun this late, but I have been thinking about this since 
your first e-mail and one thing bothers me. Don't we have stabilization cycle 
for each release starting right from the release?

In my understanding this is exactly what the Stable releases Support Phase I is 
accepting bug fixes but no new features. After 6 months the release is moved to 
Phase II where only critical and security fixes are accepted; I think this is 
good example of stabilization cycle and the output is considered solid.

All concerns looked at I think the big problem really is to get the people 
working on these cycles. Perhaps we should encourage more active maintenance on 
our stable branches and see then what we can bring from that to our development 
branch expertise and knowledge wise.

While I'm not huge believer of constant agile development, this is one of those 
things that needs to be lived with and I think stable branches are our best bet 
for stabilization work (specifically when that work needs to land to master 
first). For long term refactoring I'd like to see us using more feature 
branches so we can keep doing the work without releasing it before it's done.


So, I think what confused most people in the thread is the title itself. The
title is Stabilization Cycles but it really means "clean up the mess we have in
the code base including bugs". We can't address technical debt in stable
branches other than bug fixes. Refactors are rejected and, of course, no
breaking changes allowed.

While your thoughts are correct, as we should encourage more folks to contribute
to stable branches and fix bugs there, it's true that there are not enough
people in every project to keep master moving forward and focusing on bugs for
master *and* stable branches at the same time. Ideally, most of that
"Stabilization" work should be backported to stable branches as well.

Flavio


My 2 Euro cents,
Erno


Cheers,
Flavio

--
@flaper87
Flavio Percoco


--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Thomas Goirand]

On 01/27/2016 06:20 AM, Ian Cordasco wrote:
>> In which way what you're proposing above is different from what we
>> currently have (ie: beta 1, 2 and 3)?
> 
> we'd be communicating more strenuously to consumers that this was
> really meant to be a release that would be a very very very smooth
> upgrade experience as it really focuses on stability.

That's IMO a very dangerous thing to do. These wouldn't be stable
releases without security support. Unless there's 3 times the staffing
in the stable release team, let's not do that. Though I very much
welcome anyone to give feedback on the beta releases (I'm very close
from announcing Mitaka b2 for Debian & Ubuntu btw).

We've been discussing release cadence over and over again. Some want it
quicker (mostly devs), some want it slower (mostly: ops) and always, the
consensus was that what we have right now is a good compromise.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Thomas Goirand]

On 01/21/2016 02:38 AM, Ian Cordasco wrote:
> That said, I'd like to see a different release cadence for cycles that are
> "stabilization cycles". We, as a community, are not using minor version
> numbers. During a stabilization cycle, I would like to see master be
> released around the 3 milestones as X.1.0, X.2.0, X.3.0. If we work that
> way, then we'll be able to avoid having to backport a lot of work to the
> X.0 series and while we could support X.0 series with specific backports,
> it would avoid stressing our already small stable teams.


Hi Ian,

In which way what you're proposing above is different from what we
currently have (ie: beta 1, 2 and 3)?

FYI, even though I often skip the beta 1 (because I'm still working on
the previous stable), I always release beta 2 and 3 as pre-releases for
everyone to test. These are uploaded to Debian experimental (well, when
the next stable Debian isn't frozen...), plus non-official backport
repositories for stable Debian and Ubuntu. I'd very much welcome more
people to consume that, but I haven't receive much feedback from it.

> My release
> strategy, however, may cause more stress for downstream packages
> though. It'll cause them to have to decide what and when to package
> and to be far more aware of each project's current development cycle.
> I'm not sure that's positive.

I'm not sure why you're saying this (as I probably didn't understand
your release idea), but what I can tell: don't count on downstream
distribution to double guess project statuses. It's simply impossible,
unless we spend a great amount of time communicating with each project,
which currently can't happen given the current staffing (which I know is
scarce on all downstream distros, including: Red Hat, Debian, Ubuntu and
Suse).

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Ian Cordasco]

 

-Original Message-
From: Thomas Goirand 
Reply: OpenStack Development Mailing List (not for usage questions) 

Date: January 26, 2016 at 12:48:09
To: OpenStack Development Mailing List (not for usage questions) 

Subject:  Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on 
the idea to move it forward

> On 01/21/2016 02:38 AM, Ian Cordasco wrote:
> > That said, I'd like to see a different release cadence for cycles that are
> > "stabilization cycles". We, as a community, are not using minor version
> > numbers. During a stabilization cycle, I would like to see master be
> > released around the 3 milestones as X.1.0, X.2.0, X.3.0. If we work that
> > way, then we'll be able to avoid having to backport a lot of work to the
> > X.0 series and while we could support X.0 series with specific backports,
> > it would avoid stressing our already small stable teams.
>  
>  
> Hi Ian,
>  
> In which way what you're proposing above is different from what we
> currently have (ie: beta 1, 2 and 3)?

It's different in that these wouldn't be X+1.0.0b{1,2,3} but X.{1,2,3}.0. For a 
more concrete example, it would be something like:

If Glance were to do this, for mitaka, then instead of releasing 13.0.0b{1,2,3} 
it would release 12.{1,2,3}.0 at each milestone. According to semantic 
versioning we could land features in those releases but we'd be communicating 
more strenuously to consumers that this was really meant to be a release that 
would be a very very very smooth upgrade experience as it really focuses on 
stability.

> FYI, even though I often skip the beta 1 (because I'm still working on
> the previous stable), I always release beta 2 and 3 as pre-releases for
> everyone to test. These are uploaded to Debian experimental (well, when
> the next stable Debian isn't frozen...), plus non-official backport
> repositories for stable Debian and Ubuntu. I'd very much welcome more
> people to consume that, but I haven't receive much feedback from it.

Right, I would suspect that this would be because the releases are beta 
releases and no one wants to spend the time to use/stress them too much. :-/

> > My release
> > strategy, however, may cause more stress for downstream packages
> > though. It'll cause them to have to decide what and when to package
> > and to be far more aware of each project's current development cycle.
> > I'm not sure that's positive.
>  
> I'm not sure why you're saying this (as I probably didn't understand
> your release idea), but what I can tell: don't count on downstream
> distribution to double guess project statuses. It's simply impossible,
> unless we spend a great amount of time communicating with each project,
> which currently can't happen given the current staffing (which I know is
> scarce on all downstream distros, including: Red Hat, Debian, Ubuntu and
> Suse).

Right. I know the downstream teams are small which is why I noted that having a 
release cadence of actual concrete releases on those milestones instead of beta 
releases might cause too much work. If it's one service doing that, perhaps not 
but if several do that in an even remotely synchronized manner, I can imagine 
that would introduce a lot of work.

Either way, my idea was already shot down because it would mean projects would 
be changing their release model frequently and that's not desirable.

> Cheers,
>  
> Thomas Goirand (zigo)

--  
Ian Cordasco
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 20/01/16 13:23 -0430, Flavio Percoco wrote:

Thoughts? Feedback?


Hey Folks,

Thanks a lot for the feedback. Great comments and proposals in the many replies.
I've gone through the whole thread and collected the most common feedback.
Here's the summary:

- The general idea of planning some sort of stabilization for a project is good
 but considering a cycle for it is terrible. It'd be easier if development
 cycles would be shorter but the 6-month based development cycles don't allow
 for planning this properly.

- Therefore, milestones are more likely to be good for this but there has to be
 a good plan. What will happen with on-going features? How does a project
 decide what to merge or not? Is it really going to help with reviews/bugs
 backlog? or would this just increase the bakclog?
 
- We shouldn't need any governance resolution/reference for this. Perhaps a

 chapter/section on the project team guide should do it.

- As other changes in the commuity, it'd be awesome to get feedback from a
 project doing this before we start encouraging other projects to do the same.


I'll work on adding something to the project team guide that covers the above
points.

did I miss something? Anything else that we should add and or consider?

Cheers,
Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 25/01/16 18:19 +0100, Ghe Rivero wrote:



Quoting Flavio Percoco (2016-01-25 16:06:36)

On 20/01/16 13:23 -0430, Flavio Percoco wrote:
>Thoughts? Feedback?

Hey Folks,

Thanks a lot for the feedback. Great comments and proposals in the many replies.
I've gone through the whole thread and collected the most common feedback.
Here's the summary:

- The general idea of planning some sort of stabilization for a project is good
  but considering a cycle for it is terrible. It'd be easier if development
  cycles would be shorter but the 6-month based development cycles don't allow
  for planning this properly.

- Therefore, milestones are more likely to be good for this but there has to be
  a good plan. What will happen with on-going features? How does a project
  decide what to merge or not? Is it really going to help with reviews/bugs
  backlog? or would this just increase the bakclog?


There is still a option about having a sorter development cycle. As Thierry
said in a previous message in this thread:

"It is not entirely impossible that due to
events organization we'll accidentally have a shorter cycle (say, 4
months instead of 6) in the future here and there. I could totally see
projects take advantage of such a short cycle to place a "stabilization
cycle" or another limited-feature-addition period."

Would be nice to have more info about it. I guess if he mentioned it, it's
because they foundation already faced that possibility.


Sure but I'd leave that for a separate thread.
Flavio


Ghe Rivero




- We shouldn't need any governance resolution/reference for this. Perhaps a
  chapter/section on the project team guide should do it.

- As other changes in the commuity, it'd be awesome to get feedback from a
  project doing this before we start encouraging other projects to do the same.


I'll work on adding something to the project team guide that covers the above
points.

did I miss something? Anything else that we should add and or consider?

Cheers,
Flavio

--
@flaper87
Flavio Percoco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Ghe Rivero]

Quoting Flavio Percoco (2016-01-25 16:06:36)
> On 20/01/16 13:23 -0430, Flavio Percoco wrote:
> >Thoughts? Feedback?
>
> Hey Folks,
>
> Thanks a lot for the feedback. Great comments and proposals in the many 
> replies.
> I've gone through the whole thread and collected the most common feedback.
> Here's the summary:
>
> - The general idea of planning some sort of stabilization for a project is 
> good
>   but considering a cycle for it is terrible. It'd be easier if development
>   cycles would be shorter but the 6-month based development cycles don't allow
>   for planning this properly.
>
> - Therefore, milestones are more likely to be good for this but there has to 
> be
>   a good plan. What will happen with on-going features? How does a project
>   decide what to merge or not? Is it really going to help with reviews/bugs
>   backlog? or would this just increase the bakclog?

There is still a option about having a sorter development cycle. As Thierry
said in a previous message in this thread:

"It is not entirely impossible that due to
events organization we'll accidentally have a shorter cycle (say, 4
months instead of 6) in the future here and there. I could totally see
projects take advantage of such a short cycle to place a "stabilization
cycle" or another limited-feature-addition period."

Would be nice to have more info about it. I guess if he mentioned it, it's
because they foundation already faced that possibility.

Ghe Rivero



> - We shouldn't need any governance resolution/reference for this. Perhaps a
>   chapter/section on the project team guide should do it.
>
> - As other changes in the commuity, it'd be awesome to get feedback from a
>   project doing this before we start encouraging other projects to do the 
> same.
>
>
> I'll work on adding something to the project team guide that covers the above
> points.
>
> did I miss something? Anything else that we should add and or consider?
>
> Cheers,
> Flavio
>
> --
> @flaper87
> Flavio Percoco
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Jeremy Stanley]

On 2016-01-22 13:54:22 +0100 (+0100), Thierry Carrez wrote:
[...]
> As a lot of people said, ideally you would add features and repay
> technical debt continuously.
[...]

If we're pulling ideals into this, ideally we wouldn't _consider_
new features until we've reasonably stabilized the ones we already
added (while riding in on magic rainbow-vomiting unicorns, perhaps).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[James Bottomley]

On Fri, 2016-01-22 at 13:54 +0100, Thierry Carrez wrote:
> Zane Bitter wrote:
> > [...] Honestly, it
> > sounds like the kind of thing you come up with when you've given
> > up.
> 
> I tend to agree with that... I think healthy projects should
> naturally 
> come up with bursts of feature addition and bursts of repaying
> technical 
> debt. That is why I prefer not to be too prescriptive here.
> 
> If some people are under the impression that pausing feature addition
> in 
> order to focus on stability is discouraged, we should fix that (and I
> think this thread is a great way of making it clearer). But I would
> be 
> reluctant to start standardizing what a "stabilization period" is, or
> starting to impose them. As a lot of people said, ideally you would
> add 
> features and repay technical debt continuously. Imposing specific 
> periods of stabilization prevents reaching that ideal state.
> 
> So in summary: yes to expressing that it is fine to have them, no to 
> being more prescriptive about them.

Experience with Linux shows that deliberate stabilisation cycles, where
you refuse to accept features simply don't work: everyone gets
frustrated and development suffers.  You get lots of features disguised
as bug fixes and a lot of fighting when you try to police the bug fixes
to extract the hidden features which saps effort.

I still think what OpenStack actually needs is simply a longer
stabilisation time.  Right now, in the 6 month cycle, there are about
five months of development beginning with the design summit and one
month of -rc stabilisation.  In today's model, to extend stabilisation
you have to steal time from feature development, which again causes a
lot of argument.  To fix this, I'd turn that around and make it one
month of feature merging followed by five months of -rc stabilisation. 
 Essentially following the merge window pattern that makes the Linux
Kernel work so well.

The behaviour changes that would have to happen would be that there
would need to be a next-openstack tree that features get landed in
which would be where stuff was pulled from during the merge window
provided they pass the usual continuous tests.  Probably the design
summit would have to be just after the end of the merge window (and the
beginning of the -rc cycle) to be effective at discussing all the new
features and any feature that failed to make it to the merge window
would get held off to the next release.  This gives an effective five
month window to land features for the next release plus a five month
stabilisation cycle while keeping a 6 month release cycle.  The cost
for doing this is largely borne by feature developers who have to plan
which merge window to aim for.

James


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[John Garbutt]

On 22 January 2016 at 02:38, Robert Collins  wrote:
> On 21 January 2016 at 07:38, Ian Cordasco  wrote:
>>
>> I think this is a solid proposal but I'm not sure what (if anything) the TC 
>> needs to do about this. This is something most non-corporate open source 
>> projects do (and even some corporate open source projects). It's the natural 
>> life-cycle of any software project (that we ship a bunch of things and then 
>> focus on stability). Granted, I haven't seen much of a focus on it in 
>> OpenStack but that's a different story.
>>
>> That said, I'd like to see a different release cadence for cycles that are 
>> "stabilization cycles". We, as a community, are not using minor version 
>> numbers. During a stabilization cycle, I would like to see master be 
>> released around the 3 milestones as X.1.0, X.2.0, X.3.0. If we work that 
>> way, then we'll be able to avoid having to backport a lot of work to the X.0 
>> series and while we could support X.0 series with specific backports, it 
>> would avoid stressing our already small stable teams. My release strategy, 
>> however, may cause more stress for downstream packages though. It'll cause 
>> them to have to decide what and when to package and to be far more aware of 
>> each project's current development cycle. I'm not sure that's positive.
>
> So the reason this was on my todo this cycle - and I'm so glad Flavio
> has picked it up (point 9 of
> https://rbtcollins.wordpress.com/2015/11/02/openstack-mitaka-debrief/)
> - was that during the Tokyo summit, in multiple sessions, folk were
> saying that they wanted space from features, to consolidate already
> added things, and to cleanup accrued debt, and that without TC
> support, they couldn't sell it back to their companies.
>
> Essentially, if the TC provides some leadership here: maybe as little as:
>  - its ok to do it [we think it will benefit our users]
>  - sets some basic expectations
>
> And then individual projects decide to do it (whether thats a PTL
> call, a vote, core consensus, whatever) - then developers have a
> platform to say to their organisation that the focus is X, don't
> expect features to land - and that they are *expected* to help with
> the cycle.
>
> Without some framework, we're leaving those developers out in the cold
> trying to explain what-and-why-and-how all by themselves.

+1 on the need to fix the "big issues" facing projects.
By "big issues" I mean problems that affect almost all users.
Stability and Technical Debt are just two "big issues".
For me, doing bug triage, code reviews, fixing the gate are all in that list.

In Nova we are trying to dedicate time in every cycle to the big
issues facing the project. We do that using by picking priorities, and
the non-priority feature freeze. There is a very rough write up on
that process from liberty here:
http://docs.openstack.org/developer/nova/process.html#non-priority-feature-freeze

It has allowed us to get rolling upgrades working and evolve our API
to do micro-versioning. It has also lead to a big backlog of other
code people want to push, lots of it already in production. There are
other issues here, and lots of ideas on the table, but lets not go
down that rabbit hole on email.

+1 for having a TC statement to set exceptions.
That should help developers unable to get permission.
Hopefully developers will also get asked to work on these things.

+1 documenting common patterns for projects to adopt.

-1 for making projects to adopt a "stability" cycle.
Feels better as a suggested pattern.
(Although it feels a little like an anti-pattern)

Thanks,
johnthetubaguy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Thierry Carrez]

Zane Bitter wrote:

[...] Honestly, it
sounds like the kind of thing you come up with when you've given up.


I tend to agree with that... I think healthy projects should naturally 
come up with bursts of feature addition and bursts of repaying technical 
debt. That is why I prefer not to be too prescriptive here.


If some people are under the impression that pausing feature addition in 
order to focus on stability is discouraged, we should fix that (and I 
think this thread is a great way of making it clearer). But I would be 
reluctant to start standardizing what a "stabilization period" is, or 
starting to impose them. As a lot of people said, ideally you would add 
features and repay technical debt continuously. Imposing specific 
periods of stabilization prevents reaching that ideal state.


So in summary: yes to expressing that it is fine to have them, no to 
being more prescriptive about them.


Regards,

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Daniel P. Berrange]

On Wed, Jan 20, 2016 at 01:23:02PM -0430, Flavio Percoco wrote:
> Greetings,
> 
> At the Tokyo summit, we discussed OpenStack's development themes in a
> cross-project session. In this session a group of folks started discussing 
> what
> topics the overall community could focus on as a shared effort. One of the
> things that was raised during this session is the need of having cycles to
> stabilize projects. This was brought up by Robert Collins again in a 
> meeting[0]
> the TC had right after the summit and no much has been done ever since.
> 
> Now, "stabilization Cycles" are easy to dream about but really hard to do and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a 
> stabilization
> cycle could bring but bear in mind that the lists below are not exhaustive. In
> fact, I'd love for other folks to chime in and help building a case in favor 
> or
> against this.
> 
> Negative(?) effects
> ===
> 
> - Project won't get new features for a period of time Economic impact on
>  developers(?)
> - It was mentioned that some folks receive bonuses for landed features
> - Economic impact on companies/market because no new features were added (?)
> - (?)

It will push more development into non-upstream vendor private
branches.

> 
> Positive effects
> 
> 
> - Focus on bug fixing
> - Reduce review backlog
> - Refactor *existing* code/features with cleanups
> - Focus on multi-cycle features (if any) and complete those
> - (?)

I don't think the idea of stabalization cycles would really have
such a positive effect, certainly not while our release cycle is
6 months in length.

If you say the next cycle is primarily stabalization, then what
you are in effect saying is that people have to wait 12 months
for their desired new feature.  In the fast moving world of
cloud, I don't think that is a very credible approach. Even
with our current workflow, where we selectively approve features
for cycles, we have this impact of forcing people to wait 12
months, or more, for their features.

In the non-stabalization cycle, we're not going to be able to
merge a larger number of features than we already do today.
So in effect we'll have 2 cycles worth of features being
proposed for 1 cycle. When we inevitably reject moany of
those features they'll have to wait for the next non-stabalization
cycle, which means 18-24 months delay.

Of course in reality this kind of delay won't happen. What will
instead happen is that various vendors will get pressure from
their customers/partners and their local branches of openstack
packages will fork & diverge even further from upstream than
they already do today.

So while upstream branch will be "stabalized", most users will
probably get a *less* stable release because they'll be using
a branch from vendors with a tonne of non-upstream stuff added.


In addition having a stablization cycle will give the impression
that the following cycle is a non-stable one and likely cause
more distruption by pushing lots of features in at one time.
Instead of having a master branch which has an approximately
constant level of stabalization, you'll create a situation
where it fluctuates significantly, which is clearly worse for
people doing continuous deployment.

I think it is important to have the mindset that master should
*always* be considered stable - we already have this in general
and it is one of the success points of openstack's development
model IMHO. The idea of stabalization cycles is a step backwards

I still believe that if you want to improve stabality of the
codebase, we'd be better off moving to a shorter development
cycle. Even the 6 month cycle we have today is quite "lumpy"
in terms of what kind of work happens from month to month. If
we moved to a 2 month cycle, I think it would relieve pressure
to push in features quickly before freeze, because people would
know they'd have another opportunity very soon, instead of having
to wait 6+ months. I've previously suggested that here:

  http://lists.openstack.org/pipermail/openstack-dev/2015-February/057614.html

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Chris Dent]

On Wed, 20 Jan 2016, Flavio Percoco wrote:


- It was mentioned that some folks receive bonuses for landed features


In this thread we've had people recoil in shock at this ^ one...


- Economic impact on companies/market because no new features were added (?)


...but I have to say it was this ^ one that gave me the most concern.

At the opensource project level I really don't think this should be
something we're actively worrying about. What we should be worrying
about is if OpenStack is any good. Often "good" will include features,
but not all the time.

Let the people doing the selling worry about the market, if they
want. That stuff is, or at least should be, on the other side of a
boundary.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 21/01/16 12:00 +0100, Julien Danjou wrote:

On Wed, Jan 20 2016, Flavio Percoco wrote:

Hi fellows,


Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.


[…]

I don't think this is a bad idea per say – obviously, who would think
it's a bad idea to fix bugs. But I'm still concerned. Isn't this in some
way just a band-aid?

If a project needs to spend an entire cycle (6 months) doing
stabilization, this tells me that its development model and operating is
having some problems. What about talking about those and trying to fix
them? Maybe we should try to fix (or at least enhance) the root cause(s)
rather than just the symptoms?

So can someone enlighten me on why some projects need an entire cycle to
work fixing bugs? :)


So, I don't think it has to be the entire cycle. It could also be a couple of
milestones (or even just 1). Thing is, I believe this has to be communicated and
I want teams to know this is fine and they are encouraged to do so.

Tl;DR: It's fine to tell folks no new features will land on this and the
upcoming milestone because they'll be used to stabilize the project.

Unfortunately, just talking and proposing to fix them doesn't help. We don't
control contributor's management and we can't make calls for them other than
proposing things. I'm not saying this will fix that issue but at least it'll
communicate properly that that will be the only way to contribute to project X
in that period of time.

Flavio


Best,
--
Julien Danjou
/* Free Software hacker
  https://julien.danjou.info */




--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Julien Danjou]

On Thu, Jan 21 2016, Flavio Percoco wrote:

> So, I don't think it has to be the entire cycle. It could also be a couple of
> milestones (or even just 1). Thing is, I believe this has to be communicated 
> and
> I want teams to know this is fine and they are encouraged to do so.
>
> Tl;DR: It's fine to tell folks no new features will land on this and the
> upcoming milestone because they'll be used to stabilize the project.

I can understand that, though I think it's a very naive approach. If
your project built technical debt for the last N cycles, unfortunately I
doubt that stating you're gonna work for ⅓ of a cycle on reducing it is
going to improve your project on the long run – that's why I was saying
"band-aid".

I'd be more inclined to spend time trying to fix the root cause that
pushes projects on the slope of the technical debt rate increase.

> Unfortunately, just talking and proposing to fix them doesn't help. We don't
> control contributor's management and we can't make calls for them other than
> proposing things. I'm not saying this will fix that issue but at least it'll
> communicate properly that that will be the only way to contribute to project X
> in that period of time.

Yes, exactly. So it's my view¹ that people will just do something else
for 1.5 month (e.g. work downstream, take vacation…), and then come back
knocking at your door for their feature to be merged, now that this
stabilization period is over. And even in the best case scenario, you'll
merge some fixes and improvement, and that's it: in the end you'll end
up with the same problems in N cycle, and you'll have to redo that
again.

That's why I'm talking about fixing the root causes. :-)

Cheers,

¹  pessimistic or realistic, YMMV :-)

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 21/01/16 11:55 +0100, Thierry Carrez wrote:

Flavio Percoco wrote:

[...]
So, the above sounds quite vague, still but that's the idea. This email
is not a formal proposal but a starting point to move this conversation forward.
Is this something other teams would be interested in? Is this something some
teams would be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one.


I think "stabilization cycles" will come in all shapes and form, so 
it's hard to standardize them (and provides little value). They will 
mean different things and happen at different times for every project, 
and that is fine.


As you said, projects can already decide to restrict feature 
development in a given cycle, so this is nothing new. We only need to 
communicate more aggressively that it is perfectly fine (and even 
encouraged) to define the amount of feature work that is acceptable 
for a project for a given cycle.


++

Precisely my point. If there's a way, from a governance perspective, to help
communicate and encourage folks to do this, I wan't to take it. It was mentioned
that some teams didn't know this was possible others that felt it was going to
be really hard w/o any support from the governance team, hence this email and
effort.


For example, we would
have to formalize how projects announce they want to have a
stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).


While I understand the value of announcing this "cycle feature 
strategy" beforehand, this comes slightly at odds with our PTL 
rotation every cycle: one PTL would announce a more stable cycle and 
then the next one would have to execute on a choice that may not be 
his.


I actually wouldn't mind if that feature strategy decision was part of 
the PTL election platform -- it sounds like that would trigger 
interesting discussions.



I wouldn't mind it either. However, I'd like us to stop having such a hard
separation between current PTL's plans and future PTL's. As a community, I'd
like us to work harder on building new leaders and a better community. I'd like
current PTLs to find in the community who would like to run for the PTL role
(regardless of the current PTL planning re-election) and work with them towards
having a better long term plan for the project. We should really stop thinking
that our responsibilities as PTLs start when the cycle begins and end when our
term ends. At least, I don't believe that.

That is to say, I'd like these plans to be discussed with the community in
advance because I believe the project will benefit from proper communication.

If I run for PTL and win because I proposed a stabilization cycle, I might end
up with a good plan and no ppl due to their tasks being re-scheduled because
their management doesn't want them to spend so much time "just fixing bugs".


Thoughts? Feedback?


Just an additional thought. It is not entirely impossible that due to 
events organization we'll accidentally have a shorter cycle (say, 4 
months instead of 6) in the future here and there. I could totally see 
projects take advantage of such a short cycle to place a 
"stabilization cycle" or another limited-feature-addition period.


++

As I mentioned in a previous reply, I think projects could also have a
stabilization milestone rather than a full cycle.

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Julien Danjou]

On Wed, Jan 20 2016, Flavio Percoco wrote:

Hi fellows,

> Now, "stabilization Cycles" are easy to dream about but really hard to do and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a 
> stabilization
> cycle could bring but bear in mind that the lists below are not exhaustive. In
> fact, I'd love for other folks to chime in and help building a case in favor 
> or
> against this.

[…]

I don't think this is a bad idea per say – obviously, who would think
it's a bad idea to fix bugs. But I'm still concerned. Isn't this in some
way just a band-aid?

If a project needs to spend an entire cycle (6 months) doing
stabilization, this tells me that its development model and operating is
having some problems. What about talking about those and trying to fix
them? Maybe we should try to fix (or at least enhance) the root cause(s)
rather than just the symptoms?

So can someone enlighten me on why some projects need an entire cycle to
work fixing bugs? :)

Best,
-- 
Julien Danjou
/* Free Software hacker
   https://julien.danjou.info */


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 21/01/16 11:22 +, Daniel P. Berrange wrote:

On Wed, Jan 20, 2016 at 01:23:02PM -0430, Flavio Percoco wrote:

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
 developers(?)
- It was mentioned that some folks receive bonuses for landed features
- Economic impact on companies/market because no new features were added (?)
- (?)


It will push more development into non-upstream vendor private
branches.



Positive effects


- Focus on bug fixing
- Reduce review backlog
- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)


I don't think the idea of stabalization cycles would really have
such a positive effect, certainly not while our release cycle is
6 months in length.

If you say the next cycle is primarily stabalization, then what
you are in effect saying is that people have to wait 12 months
for their desired new feature.  In the fast moving world of
cloud, I don't think that is a very credible approach. Even
with our current workflow, where we selectively approve features
for cycles, we have this impact of forcing people to wait 12
months, or more, for their features.


++

This is one of the main concerns and perhaps the reason why I don't think it
should be all-or-nothing. It should be perfectly fine for teams to have
stabilization milestones, FWIW.


In the non-stabalization cycle, we're not going to be able to
merge a larger number of features than we already do today.
So in effect we'll have 2 cycles worth of features being
proposed for 1 cycle. When we inevitably reject moany of
those features they'll have to wait for the next non-stabalization
cycle, which means 18-24 months delay.

Of course in reality this kind of delay won't happen. What will
instead happen is that various vendors will get pressure from
their customers/partners and their local branches of openstack
packages will fork & diverge even further from upstream than
they already do today.

So while upstream branch will be "stabalized", most users will
probably get a *less* stable release because they'll be using
a branch from vendors with a tonne of non-upstream stuff added.



I would expect these vendors to (slowly?) push their changes upstream. It'd take
time but it should certainly happen.


In addition having a stablization cycle will give the impression
that the following cycle is a non-stable one and likely cause
more distruption by pushing lots of features in at one time.
Instead of having a master branch which has an approximately
constant level of stabalization, you'll create a situation
where it fluctuates significantly, which is clearly worse for
people doing continuous deployment.

I think it is important to have the mindset that master should
*always* be considered stable - we already have this in general
and it is one of the success points of openstack's development
model IMHO. The idea of stabalization cycles is a step backwards


Perhaps, it is being presented the wrong way. I guess the main point here is how
ca we communicate that we'd like to take some time to clean-up the mess we have
in some projects. How can projects ask their team to put more efforts on
tackling technical debt rather than pushing the new sexy thing?

I could consider Mitaka as a stabilization cycle for Glance (except for the
upload path refactor spec). The team has spent quite some time on working out a
way to improve that workflow. Few other specs have been implemented but nothing
major, TBH (talking about Glance here, not the other components).

What I mean is, that I don't consider a stabilization cycle a full heads-down on
bug fixing cyle but rather a cycle where no major features are approved. What
unfortunatelly happens when these kind of cycles are announced or planned is
that contributions vanish and they are routed to places where new features land.
That should perhaps be an indicator of how good/bad these cycles are. *shurgs*


I still believe that if you want to improve stabality of the
codebase, we'd be better off moving to a 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Thierry Carrez]

Flavio Percoco wrote:

[...]
So, the above sounds quite vague, still but that's the idea. This email
is not a formal proposal but a starting point to move this conversation forward.
Is this something other teams would be interested in? Is this something some
teams would be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one.


I think "stabilization cycles" will come in all shapes and form, so it's 
hard to standardize them (and provides little value). They will mean 
different things and happen at different times for every project, and 
that is fine.


As you said, projects can already decide to restrict feature development 
in a given cycle, so this is nothing new. We only need to communicate 
more aggressively that it is perfectly fine (and even encouraged) to 
define the amount of feature work that is acceptable for a project for a 
given cycle.



For example, we would
have to formalize how projects announce they want to have a
stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).


While I understand the value of announcing this "cycle feature strategy" 
beforehand, this comes slightly at odds with our PTL rotation every 
cycle: one PTL would announce a more stable cycle and then the next one 
would have to execute on a choice that may not be his.


I actually wouldn't mind if that feature strategy decision was part of 
the PTL election platform -- it sounds like that would trigger 
interesting discussions.



Thoughts? Feedback?


Just an additional thought. It is not entirely impossible that due to 
events organization we'll accidentally have a shorter cycle (say, 4 
months instead of 6) in the future here and there. I could totally see 
projects take advantage of such a short cycle to place a "stabilization 
cycle" or another limited-feature-addition period.


Regards,

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Ryan Brown]

On 01/21/2016 06:23 AM, Chris Dent wrote:

On Wed, 20 Jan 2016, Flavio Percoco wrote:


- It was mentioned that some folks receive bonuses for landed features


In this thread we've had people recoil in shock at this ^ one...


- Economic impact on companies/market because no new features were
added (?)


...but I have to say it was this ^ one that gave me the most concern.

At the opensource project level I really don't think this should be
something we're actively worrying about. What we should be worrying
about is if OpenStack is any good. Often "good" will include features,
but not all the time.

Let the people doing the selling worry about the market, if they
want. That stuff is, or at least should be, on the other side of a
boundary.


I'm certain that they will worry about the market.

But look at where contributions come from. A glance at stackalytics says 
that only 11% of contributors are independent, meaning companies are 89% 
of the contributions. Whether we acknowledge it at the project level or 
not, features and "the OpenStack market" are going to be a priority for 
a some portion of those 89% of contributions.


Those contributors also want openstack to be "good" but they also have 
deadlines to meet internally. Having a freeze upstream for stabilization 
is going to put downstream development into overdrive, no doubt. That 
would be a poor precedent to have set given where the bulk of 
contributions come from.


--
Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Thierry Carrez]

Flavio Percoco wrote:

On 21/01/16 11:55 +0100, Thierry Carrez wrote:

As you said, projects can already decide to restrict feature
development in a given cycle, so this is nothing new. We only need to
communicate more aggressively that it is perfectly fine (and even
encouraged) to define the amount of feature work that is acceptable
for a project for a given cycle.


++

Precisely my point. If there's a way, from a governance perspective, to help
communicate and encourage folks to do this, I wan't to take it. It was mentioned
that some teams didn't know this was possible others that felt it was going to
be really hard w/o any support from the governance team, hence this email and
effort.


The light approach would be to document the possibility in the project 
team guide. The heavy approach would be to take a TC resolution. Both 
solutions would have to be mentioned on openstack-dev and the weekly 
digest to get extra publicity.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 21/01/16 16:50 +0100, Thierry Carrez wrote:

Flavio Percoco wrote:

On 21/01/16 11:55 +0100, Thierry Carrez wrote:

As you said, projects can already decide to restrict feature
development in a given cycle, so this is nothing new. We only need to
communicate more aggressively that it is perfectly fine (and even
encouraged) to define the amount of feature work that is acceptable
for a project for a given cycle.


++

Precisely my point. If there's a way, from a governance perspective, to help
communicate and encourage folks to do this, I wan't to take it. It was mentioned
that some teams didn't know this was possible others that felt it was going to
be really hard w/o any support from the governance team, hence this email and
effort.


The light approach would be to document the possibility in the project 
team guide. The heavy approach would be to take a TC resolution. Both 
solutions would have to be mentioned on openstack-dev and the weekly 
digest to get extra publicity.


I'm aiming for the heavy approach here and, if it is not redundant, the light
one as well.

Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Markus Zoeller]

Flavio Percoco  wrote on 01/21/2016 09:13:02 AM:

> From: Flavio Percoco 
> To: "Daniel P. Berrange" 
> Cc: "OpenStack Development Mailing List \(not for usage questions\)" 
> 
> Date: 01/21/2016 01:47 PM
> Subject: Re: [openstack-dev] [all][tc] Stabilization cycles: 
> Elaborating on the idea to move it forward
> 
> On 21/01/16 11:22 +, Daniel P. Berrange wrote:
> >On Wed, Jan 20, 2016 at 01:23:02PM -0430, Flavio Percoco wrote:
> >> Greetings,
> >>
> >> At the Tokyo summit, we discussed OpenStack's development themes in a
> >> cross-project session. In this session a group of folks started 
> discussing what
> >> topics the overall community could focus on as a shared effort. One 
of the
> >> things that was raised during this session is the need of having 
cycles to
> >> stabilize projects. This was brought up by Robert Collins again in 
> a meeting[0]
> >> the TC had right after the summit and no much has been done ever 
since.
> >>
> >> Now, "stabilization Cycles" are easy to dream about but really hardto 
do and
> >> enforce. Nonetheless, they are still worth a try or, at the very 
least, a
> >> thought. I'll try to go through some of the issues and benefits a 
> stabilization
> >> cycle could bring but bear in mind that the lists below are not 
> exhaustive. In
> >> fact, I'd love for other folks to chime in and help building a case
> in favor or
> >> against this.
> >>
> >> Negative(?) effects
> >> ===
> >>
> >> - Project won't get new features for a period of time Economic impact 
on
> >>  developers(?)
> >> - It was mentioned that some folks receive bonuses for landed 
features
> >> - Economic impact on companies/market because no new features were 
added (?)
> >> - (?)
> >
> >It will push more development into non-upstream vendor private
> >branches.
> >
> >>
> >> Positive effects
> >> 
> >>
> >> - Focus on bug fixing
> >> - Reduce review backlog
> >> - Refactor *existing* code/features with cleanups
> >> - Focus on multi-cycle features (if any) and complete those
> >> - (?)
> >
> >I don't think the idea of stabalization cycles would really have
> >such a positive effect, certainly not while our release cycle is
> >6 months in length.
> >
> >If you say the next cycle is primarily stabalization, then what
> >you are in effect saying is that people have to wait 12 months
> >for their desired new feature.  In the fast moving world of
> >cloud, I don't think that is a very credible approach. Even
> >with our current workflow, where we selectively approve features
> >for cycles, we have this impact of forcing people to wait 12
> >months, or more, for their features.
> 
> ++
> 
> This is one of the main concerns and perhaps the reason why I don't 
think it
> should be all-or-nothing. It should be perfectly fine for teams to have
> stabilization milestones, FWIW.
> 
> >In the non-stabalization cycle, we're not going to be able to
> >merge a larger number of features than we already do today.
> >So in effect we'll have 2 cycles worth of features being
> >proposed for 1 cycle. When we inevitably reject moany of
> >those features they'll have to wait for the next non-stabalization
> >cycle, which means 18-24 months delay.
> >
> >Of course in reality this kind of delay won't happen. What will
> >instead happen is that various vendors will get pressure from
> >their customers/partners and their local branches of openstack
> >packages will fork & diverge even further from upstream than
> >they already do today.
> >
> >So while upstream branch will be "stabalized", most users will
> >probably get a *less* stable release because they'll be using
> >a branch from vendors with a tonne of non-upstream stuff added.
> >
> 
> I would expect these vendors to (slowly?) push their changes 
upstream.It'd take
> time but it should certainly happen.
> 
> >In addition having a stablization cycle will give the impression
> >that the following cycle is a non-stable one and likely cause
> >more distruption by pushing lots of features in at one time.
> >Instead of having a master branch which has an approximately
> >constant level of stabalization, you'll create a situation
> >where it fluctuates significantly, which is clearly worse for
> >people doing continuous deployment.
> >
> >I think it is important to have the mindset that master should
> >*always* be considered stable - we already have this in general
> >and it is one of the success points of openstack's development
> >model IMHO. The idea of stabalization cycles is a step backwards
> 
> Perhaps, it is being presented the wrong way. I guess the main point 
here is how
> ca we communicate that we'd like to take some time to clean-up the mess 
we have
> in some projects. How can projects ask their team to put more efforts on
> tackling technical debt rather than pushing the new sexy thing?
> 
> I could consider Mitaka as a stabilization cycle 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Doug Hellmann]

Excerpts from Markus Zoeller's message of 2016-01-21 18:37:00 +0100:
> Flavio Percoco  wrote on 01/21/2016 09:13:02 AM:
> 
> > From: Flavio Percoco 
> > To: "Daniel P. Berrange" 
> > Cc: "OpenStack Development Mailing List \(not for usage questions\)" 
> > 
> > Date: 01/21/2016 01:47 PM
> > Subject: Re: [openstack-dev] [all][tc] Stabilization cycles: 
> > Elaborating on the idea to move it forward
> > 
> > On 21/01/16 11:22 +, Daniel P. Berrange wrote:
> > >On Wed, Jan 20, 2016 at 01:23:02PM -0430, Flavio Percoco wrote:
> > >> Greetings,
> > >>
> > >> At the Tokyo summit, we discussed OpenStack's development themes in a
> > >> cross-project session. In this session a group of folks started 
> > discussing what
> > >> topics the overall community could focus on as a shared effort. One 
> of the
> > >> things that was raised during this session is the need of having 
> cycles to
> > >> stabilize projects. This was brought up by Robert Collins again in 
> > a meeting[0]
> > >> the TC had right after the summit and no much has been done ever 
> since.
> > >>
> > >> Now, "stabilization Cycles" are easy to dream about but really hardto 
> do and
> > >> enforce. Nonetheless, they are still worth a try or, at the very 
> least, a
> > >> thought. I'll try to go through some of the issues and benefits a 
> > stabilization
> > >> cycle could bring but bear in mind that the lists below are not 
> > exhaustive. In
> > >> fact, I'd love for other folks to chime in and help building a case
> > in favor or
> > >> against this.
> > >>
> > >> Negative(?) effects
> > >> ===
> > >>
> > >> - Project won't get new features for a period of time Economic impact 
> on
> > >>  developers(?)
> > >> - It was mentioned that some folks receive bonuses for landed 
> features
> > >> - Economic impact on companies/market because no new features were 
> added (?)
> > >> - (?)
> > >
> > >It will push more development into non-upstream vendor private
> > >branches.
> > >
> > >>
> > >> Positive effects
> > >> 
> > >>
> > >> - Focus on bug fixing
> > >> - Reduce review backlog
> > >> - Refactor *existing* code/features with cleanups
> > >> - Focus on multi-cycle features (if any) and complete those
> > >> - (?)
> > >
> > >I don't think the idea of stabalization cycles would really have
> > >such a positive effect, certainly not while our release cycle is
> > >6 months in length.
> > >
> > >If you say the next cycle is primarily stabalization, then what
> > >you are in effect saying is that people have to wait 12 months
> > >for their desired new feature.  In the fast moving world of
> > >cloud, I don't think that is a very credible approach. Even
> > >with our current workflow, where we selectively approve features
> > >for cycles, we have this impact of forcing people to wait 12
> > >months, or more, for their features.
> > 
> > ++
> > 
> > This is one of the main concerns and perhaps the reason why I don't 
> think it
> > should be all-or-nothing. It should be perfectly fine for teams to have
> > stabilization milestones, FWIW.
> > 
> > >In the non-stabalization cycle, we're not going to be able to
> > >merge a larger number of features than we already do today.
> > >So in effect we'll have 2 cycles worth of features being
> > >proposed for 1 cycle. When we inevitably reject moany of
> > >those features they'll have to wait for the next non-stabalization
> > >cycle, which means 18-24 months delay.
> > >
> > >Of course in reality this kind of delay won't happen. What will
> > >instead happen is that various vendors will get pressure from
> > >their customers/partners and their local branches of openstack
> > >packages will fork & diverge even further from upstream than
> > >they already do today.
> > >
> > >So while upstream branch will be "stabalized", most users will
> > >probably get a *less* stable release because they'll be using
> > >a branch from vendors with a tonne of non-upstream stuff added.
> > >
> > 
> > I would expect these vendors to (slowly?) push their changes 
> upstream.It'd take
> > time but it should certainly happen.
> > 
> > >In addition having a stablization cycle will give the impression
> > >that the following cycle is a non-stable one and likely cause
> > >more distruption by pushing lots of features in at one time.
> > >Instead of having a master branch which has an approximately
> > >constant level of stabalization, you'll create a situation
> > >where it fluctuates significantly, which is clearly worse for
> > >people doing continuous deployment.
> > >
> > >I think it is important to have the mindset that master should
> > >*always* be considered stable - we already have this in general
> > >and it is one of the success points of openstack's development
> > >model IMHO. The idea of stabalization cycles is a step backwards
> > 
> > Perhaps, it is being presented the wrong way. I guess 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Zane Bitter]

On 20/01/16 12:53, Flavio Percoco wrote:

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started
discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a
meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to
do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a
stabilization
cycle could bring but bear in mind that the lists below are not
exhaustive. In
fact, I'd love for other folks to chime in and help building a case in
favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
  developers(?)
- It was mentioned that some folks receive bonuses for landed features


o.O

Is this real life???


- Economic impact on companies/market because no new features were added
(?)
- (?)

Positive effects


- Focus on bug fixing


Or maybe just a focus on anything but upstream OpenStack work


- Reduce review backlog


Or increase the review backlog.

Or leave it about the same. It'll definitely be one of those.


- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)

A stabilization cycle, as it was also discussed in the aforementioned
meeting[0], doesn't need to be all or nothing. For instance, it should be
perfectly fine for a project to say that a project would dedicate 50% of
the
cycle to stabilization and the rest to complete some pending features.


I guess not being all-or-nothing is a good thing, but in that case what 
does this even mean in practice? If there's a review up for a feature 
what would you do differently under this policy? Merge half of it? Flip 
a coin and only review if it comes up heads?



Moreover,
each project is free to choose when/if a stabilization cycle would be
good for
it or not.

For example, the Glance team is currently working on refactoring the image
import workflow. This is a long term effort that will require at least 2
cycles
to be completed. Furthermore, it's very likely these changes will
introduce bugs
and that will require further work. If the Glance team would decide
(this is not
an actual proposal... yet :) to use Newton as a stabilization cycle, the
team
would be able to focus all its forces on fixing those bugs, completing the
feature and tackling other, long-term, pending issues. In the case of
Glance,
this would impact *only glance* and not other projects under the Glance
team
umbrella like glanceclient and glance_store. In fact, this would be a
perfect
time for the glance team to dedicate time to improving glanceclient and
catch up
with the server side latest changes.

So, the above sounds quite vague, still but that's the idea. This email
is not a
formal proposal but a starting point to move this conversation forward.
Is this
something other teams would be interested in? Is this something some
teams would
be entirely against? Why?


I actually hate this idea really quite a lot, largely for the same 
reasons that Julien and Dan have already articulated. Honestly, it 
sounds like the kind of thing you come up with when you've given up.


Instead a project could develop a long-term architecture plan that makes 
the features on its roadmap easier to implement in a robust way. Or 
introduce new features that simplify the code base and reduce the 
prevalence of existing bugs. Or demand working, tested, incremental 
changes instead of accepting unreviewable 5k line feature patches. Or 
invest in improving testing. Or break the project up into smaller units 
with clear API boundaries and give them specialist review teams. Or get 
a bunch of specialist exploratory testers to find bugs instead of 
waiting for them to affect developers somehow. Or... YMMV for any given 
idea on any given project, but the point is that saying "ok, no more 
features" is what you do as a last resort when you have literally zero 
ideas.


I guess it bugs me because I think it's an instance of a larger class of 
problem, which is characterised by the notion that one's future, better 
informed self will somehow make worse decisions than one's current self. 
i.e. you assume that you're getting stupider over time, so you decide to 
ignore the merits of any individual decision and substitute a default 
answer ("no") that you've formulated a priori. In a way it's the 
opposite of engineering.





 From a governance perspective, projects are already empowered to do
this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Joshua Harlow]

Julien Danjou wrote:

On Thu, Jan 21 2016, Flavio Percoco wrote:


So, I don't think it has to be the entire cycle. It could also be a couple of
milestones (or even just 1). Thing is, I believe this has to be communicated and
I want teams to know this is fine and they are encouraged to do so.

Tl;DR: It's fine to tell folks no new features will land on this and the
upcoming milestone because they'll be used to stabilize the project.


I can understand that, though I think it's a very naive approach. If
your project built technical debt for the last N cycles, unfortunately I
doubt that stating you're gonna work for ⅓ of a cycle on reducing it is
going to improve your project on the long run – that's why I was saying
"band-aid".

I'd be more inclined to spend time trying to fix the root cause that
pushes projects on the slope of the technical debt rate increase.


Unfortunately, just talking and proposing to fix them doesn't help. We don't
control contributor's management and we can't make calls for them other than
proposing things. I'm not saying this will fix that issue but at least it'll
communicate properly that that will be the only way to contribute to project X
in that period of time.


Yes, exactly. So it's my view¹ that people will just do something else
for 1.5 month (e.g. work downstream, take vacation…), and then come back
knocking at your door for their feature to be merged, now that this
stabilization period is over. And even in the best case scenario, you'll
merge some fixes and improvement, and that's it: in the end you'll end
up with the same problems in N cycle, and you'll have to redo that
again.

That's why I'm talking about fixing the root causes. :-)

Cheers,

¹  pessimistic or realistic, YMMV :-)


IMHO realistic, there are root causes that we need to dig out and fully 
expose to really deal with the issue of why stabilization cycles are 
needed in the first place (and said issues exposed there are going to be 
painful, and controversial and such, that's just how it is going to be).


Overall though, I'm glad we are talking about this and starting to think 
about what we as a community can do to start talking and thinking about 
these issues (and hopefully figuring out a plan to resolve some or all 
of those issues).


In all honesty its likely going to require a carrot and a stick (in some 
cases more of a carrot or more of a stick) to get companies that want to 
focus on features to think about focusing on stability. If done 
incorrectly this will have a real impact on some peoples lives and 
businesses (for better or worse this is the reality of the world, sorry 
for people that have just realized this, time to get some coffee for 
u...) so we really need to be thoughtful about how to go about this.


Anyways, enough rant/comments/thoughts from me, +1 for the general idea, 
and +1 for starting to think about what are the root causes of requiring 
this kind of cycle in the first place (to large of project? to many 
features? not enough contributors? to much code? to much junk/debt in 
your project that never got cleaned up/removed? ...)


-Josh




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Kashyap Chamarthy]

On Thu, Jan 21, 2016 at 06:37:00PM +0100, Markus Zoeller wrote:
> Flavio Percoco  wrote on 01/21/2016 09:13:02 AM:

[...]

First, positive remark(s): 

Thanks for writing this up.  FWIW, I support the notion of having
milestones focusing on stability, as opposed to explicitly declaring a
whole cycle as 'stable' as I agree (as do you) with Dan Berrange's
reasoning.

(Also see Doug's comment in the thread: "projects the option of choosing
a release model that allows for multiple releases per 6 month cycle,
while still maintaining a single stable release after the cycle.")

> > >> Negative(?) effects
> > >> ===
> > >>
> > >> - Project won't get new features for a period of time Economic
> > >> impact  on developers(?)
> > >> - It was mentioned that some folks receive bonuses for landed 
> > >>   features

This (non-point) reminds of a recent comment I've read elsewhere[1]
about why websites have been becoming bloated[1], and how people are
(dis)incentivized.  [NB: This was written in the context of websites;
we're talking about an Infra project, so adjust the view accordingly]:

   "[...] People (designers, coders, etc) get bonuses and paychecks for
   creating stuff more than tearing down stuff.

   Put this on your resume -- "Implemented feature x, designed y, added
   z" vs "Cut out 10k lines worth of crap only 10% of customers
   [operators] used, stripped away stupid 1Mb worth for js that displays
   animated snowflakes, etc".  You'd produce a better perception by
   claiming you added / created / built, rather than deleted.

   So it is not surprising that more stuff gets built, more code added
   to the pile, more features implemented. Heck, even GMail keeps
   changing every 6 months for apparently no reason. But in reality
   there is a reason -- Google has full time designers on the GMail
   team. There is probably no way they'd end the year with "Yap, site
   worked great, we did a nice job 2 years ago, so I didn't touch it
   this year."

[...]
 
> I try to handle in one post the different aspects which came up so
> far:
> 
> wrt dedicated stabilization cycles|milestones:
> 
> Piled up (=older) bugs are harder to solve than fresh ones.  I've
> seen next to no bug report in Nova which has all the necessary
> data to do a proper analysis. There are usually 1-3 requests to
> the bug reporter necessary to get enough data.  This makes me
> believe that stabilization should be a continuous effort.

Whole-heartedly agree with this.  It just ought to be a _continuous_
effort.

While we're at it, thanks Markus for your patient (and productive)
efforts on bug triaging in Nova!

[...]

[1] https://news.ycombinator.com/item?id=10820716

-- 
/kashyap

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Rochelle Grober]

Devananda van der Veen, on  January 21, 2016 5:14 PM wrote:

On Wed, Jan 20, 2016 at 9:53 AM, Flavio Percoco 
> wrote:
Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
 developers(?)
- It was mentioned that some folks receive bonuses for landed features
- Economic impact on companies/market because no new features were added (?)
- (?)

Positive effects


- Focus on bug fixing
- Reduce review backlog
- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)

A stabilization cycle, as it was also discussed in the aforementioned
meeting[0], doesn't need to be all or nothing. For instance, it should be
perfectly fine for a project to say that a project would dedicate 50% of the
cycle to stabilization and the rest to complete some pending features. Moreover,
each project is free to choose when/if a stabilization cycle would be good for
it or not.

For example, the Glance team is currently working on refactoring the image
import workflow. This is a long term effort that will require at least 2 cycles
to be completed. Furthermore, it's very likely these changes will introduce bugs
and that will require further work. If the Glance team would decide (this is not
an actual proposal... yet :) to use Newton as a stabilization cycle, the team
would be able to focus all its forces on fixing those bugs, completing the
feature and tackling other, long-term, pending issues. In the case of Glance,
this would impact *only glance* and not other projects under the Glance team
umbrella like glanceclient and glance_store. In fact, this would be a perfect
time for the glance team to dedicate time to improving glanceclient and catch up
with the server side latest changes.

So, the above sounds quite vague, still but that's the idea. This email is not a
formal proposal but a starting point to move this conversation forward. Is this
something other teams would be interested in? Is this something some teams would
be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one. For example, we would
have to formalize how projects announce they want to have a stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).

Thoughts? Feedback?
Flavio


Thanks for writing this up, Flavio.

The topic's come up in smaller discussion groups several times over the last 
few years, mostly with a nod to "that would be great, except the corporations 
won't let it happen".

To everyone who's replied with shock to this thread, the reality is that nearly 
all of the developer-hours which fuel OpenStack's progress are funded directly 
by corporations, whether big or small. Even those folks who have worked in open 
source for a long time, and are working on OpenStack by choice, are being paid 
by companies deeply invested in the success of this project. Some developers 
are adept at separating the demands of their employer from the best interests 
of the community. Some are not. I don't have hard data, but I suspect that most 
of the nearly-2000 developers who have contributed to OpenStack during the 
Mitaka cycle are working on what ever they're working on BECAUSE IT MATTERS TO 
THEIR EMPLOYER.

Every project experiences pressure from companies who are trying to land very 
specific features. Why? Because they're all chasing first-leader advantage in 
the market. Lots of features are in flight right now, and, even though it 
sounds unbelievable, many companies announce those features in their products 
BEFORE they actually land upstream. Crazy, right? Except... IT WORKS. Other 
companies buy their product because they are buying a PRODUCT from some 
company. It happens to contain OpenStack. And it has a bunch of unmerged 
features.

With my 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Devananda van der Veen]

On Wed, Jan 20, 2016 at 9:53 AM, Flavio Percoco  wrote:

> Greetings,
>
> At the Tokyo summit, we discussed OpenStack's development themes in a
> cross-project session. In this session a group of folks started discussing
> what
> topics the overall community could focus on as a shared effort. One of the
> things that was raised during this session is the need of having cycles to
> stabilize projects. This was brought up by Robert Collins again in a
> meeting[0]
> the TC had right after the summit and no much has been done ever since.
>
> Now, "stabilization Cycles" are easy to dream about but really hard to do
> and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a
> stabilization
> cycle could bring but bear in mind that the lists below are not
> exhaustive. In
> fact, I'd love for other folks to chime in and help building a case in
> favor or
> against this.
>
> Negative(?) effects
> ===
>
> - Project won't get new features for a period of time Economic impact on
>  developers(?)
> - It was mentioned that some folks receive bonuses for landed features
> - Economic impact on companies/market because no new features were added
> (?)
> - (?)
>
> Positive effects
> 
>
> - Focus on bug fixing
> - Reduce review backlog
> - Refactor *existing* code/features with cleanups
> - Focus on multi-cycle features (if any) and complete those
> - (?)
>
> A stabilization cycle, as it was also discussed in the aforementioned
> meeting[0], doesn't need to be all or nothing. For instance, it should be
> perfectly fine for a project to say that a project would dedicate 50% of
> the
> cycle to stabilization and the rest to complete some pending features.
> Moreover,
> each project is free to choose when/if a stabilization cycle would be good
> for
> it or not.
>
> For example, the Glance team is currently working on refactoring the image
> import workflow. This is a long term effort that will require at least 2
> cycles
> to be completed. Furthermore, it's very likely these changes will
> introduce bugs
> and that will require further work. If the Glance team would decide (this
> is not
> an actual proposal... yet :) to use Newton as a stabilization cycle, the
> team
> would be able to focus all its forces on fixing those bugs, completing the
> feature and tackling other, long-term, pending issues. In the case of
> Glance,
> this would impact *only glance* and not other projects under the Glance
> team
> umbrella like glanceclient and glance_store. In fact, this would be a
> perfect
> time for the glance team to dedicate time to improving glanceclient and
> catch up
> with the server side latest changes.
>
> So, the above sounds quite vague, still but that's the idea. This email is
> not a
> formal proposal but a starting point to move this conversation forward. Is
> this
> something other teams would be interested in? Is this something some teams
> would
> be entirely against? Why?
>
> From a governance perspective, projects are already empowered to do this
> and
> they don't (and won't) need to be granted permission to have stabilization
> cycles. However, the TC could work on formalizing this process so that
> teams
> have a reference to follow when they want to have one. For example, we
> would
> have to formalize how projects announce they want to have a stabilization
> cycle
> (I believe it should be done before the mid-term of the ongoing cycle).
>
> Thoughts? Feedback?
> Flavio
>
>

Thanks for writing this up, Flavio.

The topic's come up in smaller discussion groups several times over the
last few years, mostly with a nod to "that would be great, except the
corporations won't let it happen".

To everyone who's replied with shock to this thread, the reality is that
nearly all of the developer-hours which fuel OpenStack's progress are
funded directly by corporations, whether big or small. Even those folks who
have worked in open source for a long time, and are working on OpenStack by
choice, are being paid by companies deeply invested in the success of this
project. Some developers are adept at separating the demands of their
employer from the best interests of the community. Some are not. I don't
have hard data, but I suspect that most of the nearly-2000 developers who
have contributed to OpenStack during the Mitaka cycle are working on what
ever they're working on BECAUSE IT MATTERS TO THEIR EMPLOYER.

Every project experiences pressure from companies who are trying to land
very specific features. Why? Because they're all chasing first-leader
advantage in the market. Lots of features are in flight right now, and,
even though it sounds unbelievable, many companies announce those features
in their products BEFORE they actually land upstream. Crazy, right?
Except... IT WORKS. Other companies buy their product because they are
buying a PRODUCT from some company. It happens to 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Robert Collins]

On 21 January 2016 at 07:38, Ian Cordasco  wrote:
>
> I think this is a solid proposal but I'm not sure what (if anything) the TC 
> needs to do about this. This is something most non-corporate open source 
> projects do (and even some corporate open source projects). It's the natural 
> life-cycle of any software project (that we ship a bunch of things and then 
> focus on stability). Granted, I haven't seen much of a focus on it in 
> OpenStack but that's a different story.
>
> That said, I'd like to see a different release cadence for cycles that are 
> "stabilization cycles". We, as a community, are not using minor version 
> numbers. During a stabilization cycle, I would like to see master be released 
> around the 3 milestones as X.1.0, X.2.0, X.3.0. If we work that way, then 
> we'll be able to avoid having to backport a lot of work to the X.0 series and 
> while we could support X.0 series with specific backports, it would avoid 
> stressing our already small stable teams. My release strategy, however, may 
> cause more stress for downstream packages though. It'll cause them to have to 
> decide what and when to package and to be far more aware of each project's 
> current development cycle. I'm not sure that's positive.

So the reason this was on my todo this cycle - and I'm so glad Flavio
has picked it up (point 9 of
https://rbtcollins.wordpress.com/2015/11/02/openstack-mitaka-debrief/)
- was that during the Tokyo summit, in multiple sessions, folk were
saying that they wanted space from features, to consolidate already
added things, and to cleanup accrued debt, and that without TC
support, they couldn't sell it back to their companies.

Essentially, if the TC provides some leadership here: maybe as little as:
 - its ok to do it [we think it will benefit our users]
 - sets some basic expectations

And then individual projects decide to do it (whether thats a PTL
call, a vote, core consensus, whatever) - then developers have a
platform to say to their organisation that the focus is X, don't
expect features to land - and that they are *expected* to help with
the cycle.

Without some framework, we're leaving those developers out in the cold
trying to explain what-and-why-and-how all by themselves.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Steve Martinelli]

This is something I'd really like to see happen. It's an idea we've been
tossing around for the Keystone project, or at least a release with minimal
features, maybe 1 or 2. More comments in line

Steve Martinelli
Keystone PTL

Flavio Percoco  wrote on 2016/01/20 08:23:02 AM:

> From: Flavio Percoco 
> To: openstack-dev@lists.openstack.org
> Date: 2016/01/20 01:01 PM
> Subject: [openstack-dev] [all][tc] Stabilization cycles: Elaborating
> on the idea to move it forward
>
> Greetings,
>
> At the Tokyo summit, we discussed OpenStack's development themes in a
> cross-project session. In this session a group of folks started
> discussing what
> topics the overall community could focus on as a shared effort. One of
the
> things that was raised during this session is the need of having cycles
to
> stabilize projects. This was brought up by Robert Collins again in
ameeting[0]
> the TC had right after the summit and no much has been done ever since.
>
> Now, "stabilization Cycles" are easy to dream about but really hard to do
and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a
> stabilization
> cycle could bring but bear in mind that the lists below are not
exhaustive. In
> fact, I'd love for other folks to chime in and help building a case
> in favor or
> against this.
>
> Negative(?) effects
> ===
>
> - Project won't get new features for a period of time Economic impact on
>   developers(?)
> - It was mentioned that some folks receive bonuses for landed features

This is a thing?! Really? I'm very surprised about this one, features
should only be included in a project if they make sense strategically for
the project. PTLs should -2 the spec if they feel it doesn't align with the
project's direction. No?

> - Economic impact on companies/market because no new features were added
(?)

I'd argue this could be spun into a positive. We get enough grief about
difficulty and complexity of OpenStack, by focusing on paying down
technical debt, we likely going to addressing some of the issues that make
things complex. We are actually listening to feedback instead of plowing
ahead with features no one uses cause they're still on Juno.


> - (?)
>
> Positive effects
> 
>
> - Focus on bug fixing
> - Reduce review backlog
> - Refactor *existing* code/features with cleanups
> - Focus on multi-cycle features (if any) and complete those
> - (?)
>
> A stabilization cycle, as it was also discussed in the aforementioned
> meeting[0], doesn't need to be all or nothing. For instance, it should be
> perfectly fine for a project to say that a project would dedicate 50% of
the
> cycle to stabilization and the rest to complete some pending
> features. Moreover,
> each project is free to choose when/if a stabilization cycle would be
good for
> it or not.
>
> For example, the Glance team is currently working on refactoring the
image
> import workflow. This is a long term effort that will require at
> least 2 cycles
> to be completed. Furthermore, it's very likely these changes will
> introduce bugs
> and that will require further work. If the Glance team would decide
> (this is not
> an actual proposal... yet :) to use Newton as a stabilization cycle, the
team
> would be able to focus all its forces on fixing those bugs, completing
the
> feature and tackling other, long-term, pending issues. In the case of
Glance,
> this would impact *only glance* and not other projects under the Glance
team
> umbrella like glanceclient and glance_store. In fact, this would be a
perfect
> time for the glance team to dedicate time to improving glanceclient
> and catch up
> with the server side latest changes.
>
> So, the above sounds quite vague, still but that's the idea. This
> email is not a
> formal proposal but a starting point to move this conversation
> forward. Is this
> something other teams would be interested in? Is this something some
> teams would
> be entirely against? Why?
>
> From a governance perspective, projects are already empowered to do this
and
> they don't (and won't) need to be granted permission to have
stabilization
> cycles. However, the TC could work on formalizing this process so that
teams
> have a reference to follow when they want to have one. For example, we
would
> have to formalize how projects announce they want to have a
> stabilization cycle
> (I believe it should be done before the mid-term of the ongoing cycle).
>
> Thoughts? Feedback?
> Flavio
>
>
> [0] http://eavesdrop.openstack.org/meetings/tc/2015/tc.
> 2015-11-03-20.07.log.html (20:47:02)
>
> --
> @flaper87
> Flavio Percoco
> [attachment "signature.asc" deleted by Steve Martinelli/Toronto/IBM]
>
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Doug Hellmann]

Excerpts from Ian Cordasco's message of 2016-01-20 18:38:26 +:
> -Original Message-
> From: Flavio Percoco 
> Reply: Flavio Percoco , OpenStack Development Mailing List 
> (not for usage questions) 
> Date: January 20, 2016 at 11:57:56
> To: openstack-dev@lists.openstack.org 
> Subject:  [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the 
> idea to move it forward
> 
> > Greetings,
> >  
> > At the Tokyo summit, we discussed OpenStack's development themes in a
> > cross-project session. In this session a group of folks started discussing 
> > what
> > topics the overall community could focus on as a shared effort. One of the
> > things that was raised during this session is the need of having cycles to
> > stabilize projects. This was brought up by Robert Collins again in a 
> > meeting[0]
> > the TC had right after the summit and no much has been done ever since.
> >  
> > Now, "stabilization Cycles" are easy to dream about but really hard to do 
> > and
> > enforce. Nonetheless, they are still worth a try or, at the very least, a
> > thought. I'll try to go through some of the issues and benefits a 
> > stabilization
> > cycle could bring but bear in mind that the lists below are not exhaustive. 
> > In
> > fact, I'd love for other folks to chime in and help building a case in 
> > favor or
> > against this.
> >  
> > Negative(?) effects
> > ===
> >  
> > - Project won't get new features for a period of time Economic impact on
> > developers(?)
> > - It was mentioned that some folks receive bonuses for landed features
> > - Economic impact on companies/market because no new features were added (?)
> > - (?)
> >  
> > Positive effects
> > 
> >  
> > - Focus on bug fixing
> > - Reduce review backlog
> > - Refactor *existing* code/features with cleanups
> > - Focus on multi-cycle features (if any) and complete those
> > - (?)
> >  
> > A stabilization cycle, as it was also discussed in the aforementioned
> > meeting[0], doesn't need to be all or nothing. For instance, it should be
> > perfectly fine for a project to say that a project would dedicate 50% of the
> > cycle to stabilization and the rest to complete some pending features. 
> > Moreover,
> > each project is free to choose when/if a stabilization cycle would be good 
> > for
> > it or not.
> >  
> > For example, the Glance team is currently working on refactoring the image
> > import workflow. This is a long term effort that will require at least 2 
> > cycles
> > to be completed. Furthermore, it's very likely these changes will introduce 
> > bugs
> > and that will require further work. If the Glance team would decide (this 
> > is not
> > an actual proposal... yet :) to use Newton as a stabilization cycle, the 
> > team
> > would be able to focus all its forces on fixing those bugs, completing the
> > feature and tackling other, long-term, pending issues. In the case of 
> > Glance,
> > this would impact *only glance* and not other projects under the Glance team
> > umbrella like glanceclient and glance_store. In fact, this would be a 
> > perfect
> > time for the glance team to dedicate time to improving glanceclient and 
> > catch up
> > with the server side latest changes.
> >  
> > So, the above sounds quite vague, still but that's the idea. This email is 
> > not a
> > formal proposal but a starting point to move this conversation forward. Is 
> > this
> > something other teams would be interested in? Is this something some teams 
> > would
> > be entirely against? Why?
> >  
> > From a governance perspective, projects are already empowered to do this and
> > they don't (and won't) need to be granted permission to have stabilization
> > cycles. However, the TC could work on formalizing this process so that teams
> > have a reference to follow when they want to have one. For example, we would
> > have to formalize how projects announce they want to have a stabilization 
> > cycle
> > (I believe it should be done before the mid-term of the ongoing cycle).
> >  
> > Thoughts? Feedback?
> > Flavio
> >  
> >  
> > [0] 
> > http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-03-20.07.log.html
> >   
> > (20:47:02)
> 
> I think this is a solid proposal but I'm not sure what (if anything) the TC 
> needs to do about this. This is something most non-corporate open source 
> projects do (and even some corporate open source projects). It's the natural 
> life-cycle of any software project (that we ship a bunch of things and then 
> focus on stability). Granted, I haven't seen much of a focus on it in 
> OpenStack but that's a different story.
> 
> That said, I'd like to see a different release cadence for cycles that are 
> "stabilization cycles". We, as a community, are not using minor version 
> numbers. During a stabilization cycle, I would like to see master be released 
> 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Jay Pipes]

On 01/20/2016 01:30 PM, Doug Hellmann wrote:

Excerpts from Flavio Percoco's message of 2016-01-20 13:23:02 -0430:

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
   developers(?)
- It was mentioned that some folks receive bonuses for landed features


As uncomfortable as it may be to tell another contributor "no" and
realize that it will have a direct financial impact on them, we
can't encourage a contribution model in which everyone is vying for
their own special patches to land in order to get paid. That destroys
our ability to look at the overall health of the project, and
objectively examine the wisdom of adding (or removing) any given
feature. It moves the issues we have with some vendor-specific
drivers out of the driver space and into all of OpenStack, turning
our open collaboration into competition.


Amen, brother. My thoughts exactly.

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 20/01/16 18:38 +, Ian Cordasco wrote:

-Original Message-
From: Flavio Percoco 
Reply: Flavio Percoco , OpenStack Development Mailing List (not 
for usage questions) 
Date: January 20, 2016 at 11:57:56
To: openstack-dev@lists.openstack.org 
Subject:  [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the 
idea to move it forward


Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
developers(?)
- It was mentioned that some folks receive bonuses for landed features
- Economic impact on companies/market because no new features were added (?)
- (?)

Positive effects


- Focus on bug fixing
- Reduce review backlog
- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)

A stabilization cycle, as it was also discussed in the aforementioned
meeting[0], doesn't need to be all or nothing. For instance, it should be
perfectly fine for a project to say that a project would dedicate 50% of the
cycle to stabilization and the rest to complete some pending features. Moreover,
each project is free to choose when/if a stabilization cycle would be good for
it or not.

For example, the Glance team is currently working on refactoring the image
import workflow. This is a long term effort that will require at least 2 cycles
to be completed. Furthermore, it's very likely these changes will introduce bugs
and that will require further work. If the Glance team would decide (this is not
an actual proposal... yet :) to use Newton as a stabilization cycle, the team
would be able to focus all its forces on fixing those bugs, completing the
feature and tackling other, long-term, pending issues. In the case of Glance,
this would impact *only glance* and not other projects under the Glance team
umbrella like glanceclient and glance_store. In fact, this would be a perfect
time for the glance team to dedicate time to improving glanceclient and catch up
with the server side latest changes.

So, the above sounds quite vague, still but that's the idea. This email is not a
formal proposal but a starting point to move this conversation forward. Is this
something other teams would be interested in? Is this something some teams would
be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one. For example, we would
have to formalize how projects announce they want to have a stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).

Thoughts? Feedback?
Flavio


[0] http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-03-20.07.log.html
(20:47:02)


I think this is a solid proposal but I'm not sure what (if anything) the TC 
needs to do about this. This is something most non-corporate open source 
projects do (and even some corporate open source projects). It's the natural 
life-cycle of any software project (that we ship a bunch of things and then 
focus on stability). Granted, I haven't seen much of a focus on it in OpenStack 
but that's a different story.


Given the importance of this, which I believe is high, I think the TC can help
establishing some guidelines and encouraging projects to do this often enough by
expresing an opinion. There's nothing to do from a governance perspective - as I
mentioned in my first email - but I do think we need to make this a
cross-project effort, at the very least. Syncing with the release team will also
be required, especially to find a way to communicate this properly to users.

Cheers,
Flavio


That said, I'd like to see a different release cadence for cycles that are 
"stabilization cycles". We, as a community, are not using minor version 
numbers. During a stabilization cycle, I would like to see 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 20/01/16 13:44 -0500, Steve Martinelli wrote:

This is something I'd really like to see happen. It's an idea we've been
tossing around for the Keystone project, or at least a release with minimal
features, maybe 1 or 2. More comments in line

Steve Martinelli
Keystone PTL

Flavio Percoco  wrote on 2016/01/20 08:23:02 AM:


From: Flavio Percoco 
To: openstack-dev@lists.openstack.org
Date: 2016/01/20 01:01 PM
Subject: [openstack-dev] [all][tc] Stabilization cycles: Elaborating
on the idea to move it forward

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started
discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in ameeting

[0]

the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a
stabilization
cycle could bring but bear in mind that the lists below are not exhaustive.

In

fact, I'd love for other folks to chime in and help building a case
in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
  developers(?)
- It was mentioned that some folks receive bonuses for landed features


This is a thing?! Really? I'm very surprised about this one, features should
only be included in a project if they make sense strategically for the project.
PTLs should -2 the spec if they feel it doesn't align with the project's
direction. No?


The fact that people get bonuses doesn't mean the feature doesn't align with the
project. Come up w/ something useful, propose it and get it done. If it's in you
get X.

That said, I'm just guessing based on that single line that was brought up in
the meeting I linked. Whether that's true or not, we'll need someone to confirm
it (or not ;).


- Economic impact on companies/market because no new features were added (?)


I'd argue this could be spun into a positive. We get enough grief about
difficulty and complexity of OpenStack, by focusing on paying down technical
debt, we likely going to addressing some of the issues that make things
complex. We are actually listening to feedback instead of plowing ahead with
features no one uses cause they're still on Juno.


++

Cheers,
Flavio




- (?)

Positive effects


- Focus on bug fixing
- Reduce review backlog
- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)

A stabilization cycle, as it was also discussed in the aforementioned
meeting[0], doesn't need to be all or nothing. For instance, it should be
perfectly fine for a project to say that a project would dedicate 50% of the
cycle to stabilization and the rest to complete some pending
features. Moreover,
each project is free to choose when/if a stabilization cycle would be good

for

it or not.

For example, the Glance team is currently working on refactoring the image
import workflow. This is a long term effort that will require at
least 2 cycles
to be completed. Furthermore, it's very likely these changes will
introduce bugs
and that will require further work. If the Glance team would decide
(this is not
an actual proposal... yet :) to use Newton as a stabilization cycle, the team
would be able to focus all its forces on fixing those bugs, completing the
feature and tackling other, long-term, pending issues. In the case of Glance,
this would impact *only glance* and not other projects under the Glance team
umbrella like glanceclient and glance_store. In fact, this would be a perfect
time for the glance team to dedicate time to improving glanceclient
and catch up
with the server side latest changes.

So, the above sounds quite vague, still but that's the idea. This
email is not a
formal proposal but a starting point to move this conversation
forward. Is this
something other teams would be interested in? Is this something some
teams would
be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one. For example, we would
have to formalize how projects announce they want to have a
stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).

Thoughts? Feedback?
Flavio


[0] http://eavesdrop.openstack.org/meetings/tc/2015/tc.
2015-11-03-20.07.log.html (20:47:02)

--
@flaper87
Flavio Percoco
[attachment 

[openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
 developers(?)
- It was mentioned that some folks receive bonuses for landed features
- Economic impact on companies/market because no new features were added (?)
- (?)

Positive effects


- Focus on bug fixing
- Reduce review backlog
- Refactor *existing* code/features with cleanups
- Focus on multi-cycle features (if any) and complete those
- (?)

A stabilization cycle, as it was also discussed in the aforementioned
meeting[0], doesn't need to be all or nothing. For instance, it should be
perfectly fine for a project to say that a project would dedicate 50% of the
cycle to stabilization and the rest to complete some pending features. Moreover,
each project is free to choose when/if a stabilization cycle would be good for
it or not.

For example, the Glance team is currently working on refactoring the image
import workflow. This is a long term effort that will require at least 2 cycles
to be completed. Furthermore, it's very likely these changes will introduce bugs
and that will require further work. If the Glance team would decide (this is not
an actual proposal... yet :) to use Newton as a stabilization cycle, the team
would be able to focus all its forces on fixing those bugs, completing the
feature and tackling other, long-term, pending issues. In the case of Glance,
this would impact *only glance* and not other projects under the Glance team
umbrella like glanceclient and glance_store. In fact, this would be a perfect
time for the glance team to dedicate time to improving glanceclient and catch up
with the server side latest changes.

So, the above sounds quite vague, still but that's the idea. This email is not a
formal proposal but a starting point to move this conversation forward. Is this
something other teams would be interested in? Is this something some teams would
be entirely against? Why?

From a governance perspective, projects are already empowered to do this and
they don't (and won't) need to be granted permission to have stabilization
cycles. However, the TC could work on formalizing this process so that teams
have a reference to follow when they want to have one. For example, we would
have to formalize how projects announce they want to have a stabilization cycle
(I believe it should be done before the mid-term of the ongoing cycle).

Thoughts? Feedback?
Flavio


[0] 
http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-03-20.07.log.html 
(20:47:02)

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Doug Hellmann]

Excerpts from Flavio Percoco's message of 2016-01-20 13:23:02 -0430:
> Greetings,
> 
> At the Tokyo summit, we discussed OpenStack's development themes in a
> cross-project session. In this session a group of folks started discussing 
> what
> topics the overall community could focus on as a shared effort. One of the
> things that was raised during this session is the need of having cycles to
> stabilize projects. This was brought up by Robert Collins again in a 
> meeting[0]
> the TC had right after the summit and no much has been done ever since.
> 
> Now, "stabilization Cycles" are easy to dream about but really hard to do and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a 
> stabilization
> cycle could bring but bear in mind that the lists below are not exhaustive. In
> fact, I'd love for other folks to chime in and help building a case in favor 
> or
> against this.
> 
> Negative(?) effects
> ===
> 
> - Project won't get new features for a period of time Economic impact on
>   developers(?)
> - It was mentioned that some folks receive bonuses for landed features

As uncomfortable as it may be to tell another contributor "no" and
realize that it will have a direct financial impact on them, we
can't encourage a contribution model in which everyone is vying for
their own special patches to land in order to get paid. That destroys
our ability to look at the overall health of the project, and
objectively examine the wisdom of adding (or removing) any given
feature. It moves the issues we have with some vendor-specific
drivers out of the driver space and into all of OpenStack, turning
our open collaboration into competition.

> - Economic impact on companies/market because no new features were added (?)
> - (?)
> 
> Positive effects
> 
> 
> - Focus on bug fixing
> - Reduce review backlog
> - Refactor *existing* code/features with cleanups
> - Focus on multi-cycle features (if any) and complete those
> - (?)
> 
> A stabilization cycle, as it was also discussed in the aforementioned
> meeting[0], doesn't need to be all or nothing. For instance, it should be
> perfectly fine for a project to say that a project would dedicate 50% of the
> cycle to stabilization and the rest to complete some pending features. 
> Moreover,
> each project is free to choose when/if a stabilization cycle would be good for
> it or not.
> 
> For example, the Glance team is currently working on refactoring the image
> import workflow. This is a long term effort that will require at least 2 
> cycles
> to be completed. Furthermore, it's very likely these changes will introduce 
> bugs
> and that will require further work. If the Glance team would decide (this is 
> not
> an actual proposal... yet :) to use Newton as a stabilization cycle, the team
> would be able to focus all its forces on fixing those bugs, completing the
> feature and tackling other, long-term, pending issues. In the case of Glance,
> this would impact *only glance* and not other projects under the Glance team
> umbrella like glanceclient and glance_store. In fact, this would be a perfect
> time for the glance team to dedicate time to improving glanceclient and catch 
> up
> with the server side latest changes.
> 
> So, the above sounds quite vague, still but that's the idea. This email is 
> not a
> formal proposal but a starting point to move this conversation forward. Is 
> this
> something other teams would be interested in? Is this something some teams 
> would
> be entirely against? Why?
> 
> From a governance perspective, projects are already empowered to do this and
> they don't (and won't) need to be granted permission to have stabilization
> cycles. However, the TC could work on formalizing this process so that teams
> have a reference to follow when they want to have one. For example, we would
> have to formalize how projects announce they want to have a stabilization 
> cycle
> (I believe it should be done before the mid-term of the ongoing cycle).
> 
> Thoughts? Feedback?
> Flavio

I like the idea of teams taking the time to improve stability, whether
that's 100% or 50% or 10% of the time in a given cycle. I also recognize
that the community needs to make it very clear this is acceptable.

Historically, we have done a better job with big process changes like
this through experimentation, rather than trying to work them out in a
way that will work for all projects. So I think the next step is for a
project to volunteer to announce a stabilization cycle, and then we'll
work out the details for that team as they go. And after the cycle,
we'll hold a retrospective to talk about what went well and what would
need to change, so that other teams can benefit from the experience.

Doug

> 
> 
> [0] 
> http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-03-20.07.log.html 
> (20:47:02)
> 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Ian Cordasco]

-Original Message-
From: Flavio Percoco 
Reply: Flavio Percoco , OpenStack Development Mailing List 
(not for usage questions) 
Date: January 20, 2016 at 11:57:56
To: openstack-dev@lists.openstack.org 
Subject:  [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the 
idea to move it forward

> Greetings,
>  
> At the Tokyo summit, we discussed OpenStack's development themes in a
> cross-project session. In this session a group of folks started discussing 
> what
> topics the overall community could focus on as a shared effort. One of the
> things that was raised during this session is the need of having cycles to
> stabilize projects. This was brought up by Robert Collins again in a 
> meeting[0]
> the TC had right after the summit and no much has been done ever since.
>  
> Now, "stabilization Cycles" are easy to dream about but really hard to do and
> enforce. Nonetheless, they are still worth a try or, at the very least, a
> thought. I'll try to go through some of the issues and benefits a 
> stabilization
> cycle could bring but bear in mind that the lists below are not exhaustive. In
> fact, I'd love for other folks to chime in and help building a case in favor 
> or
> against this.
>  
> Negative(?) effects
> ===
>  
> - Project won't get new features for a period of time Economic impact on
> developers(?)
> - It was mentioned that some folks receive bonuses for landed features
> - Economic impact on companies/market because no new features were added (?)
> - (?)
>  
> Positive effects
> 
>  
> - Focus on bug fixing
> - Reduce review backlog
> - Refactor *existing* code/features with cleanups
> - Focus on multi-cycle features (if any) and complete those
> - (?)
>  
> A stabilization cycle, as it was also discussed in the aforementioned
> meeting[0], doesn't need to be all or nothing. For instance, it should be
> perfectly fine for a project to say that a project would dedicate 50% of the
> cycle to stabilization and the rest to complete some pending features. 
> Moreover,
> each project is free to choose when/if a stabilization cycle would be good for
> it or not.
>  
> For example, the Glance team is currently working on refactoring the image
> import workflow. This is a long term effort that will require at least 2 
> cycles
> to be completed. Furthermore, it's very likely these changes will introduce 
> bugs
> and that will require further work. If the Glance team would decide (this is 
> not
> an actual proposal... yet :) to use Newton as a stabilization cycle, the team
> would be able to focus all its forces on fixing those bugs, completing the
> feature and tackling other, long-term, pending issues. In the case of Glance,
> this would impact *only glance* and not other projects under the Glance team
> umbrella like glanceclient and glance_store. In fact, this would be a perfect
> time for the glance team to dedicate time to improving glanceclient and catch 
> up
> with the server side latest changes.
>  
> So, the above sounds quite vague, still but that's the idea. This email is 
> not a
> formal proposal but a starting point to move this conversation forward. Is 
> this
> something other teams would be interested in? Is this something some teams 
> would
> be entirely against? Why?
>  
> From a governance perspective, projects are already empowered to do this and
> they don't (and won't) need to be granted permission to have stabilization
> cycles. However, the TC could work on formalizing this process so that teams
> have a reference to follow when they want to have one. For example, we would
> have to formalize how projects announce they want to have a stabilization 
> cycle
> (I believe it should be done before the mid-term of the ongoing cycle).
>  
> Thoughts? Feedback?
> Flavio
>  
>  
> [0] 
> http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-03-20.07.log.html  
> (20:47:02)

I think this is a solid proposal but I'm not sure what (if anything) the TC 
needs to do about this. This is something most non-corporate open source 
projects do (and even some corporate open source projects). It's the natural 
life-cycle of any software project (that we ship a bunch of things and then 
focus on stability). Granted, I haven't seen much of a focus on it in OpenStack 
but that's a different story.

That said, I'd like to see a different release cadence for cycles that are 
"stabilization cycles". We, as a community, are not using minor version 
numbers. During a stabilization cycle, I would like to see master be released 
around the 3 milestones as X.1.0, X.2.0, X.3.0. If we work that way, then we'll 
be able to avoid having to backport a lot of work to the X.0 series and while 
we could support X.0 series with specific backports, it would avoid stressing 
our already small stable teams. My release strategy, however, may cause more 

Re: [openstack-dev] [all][tc] Stabilization cycles: Elaborating on the idea to move it forward

[Flavio Percoco]

On 20/01/16 14:58 -0500, Jay Pipes wrote:

On 01/20/2016 01:30 PM, Doug Hellmann wrote:

Excerpts from Flavio Percoco's message of 2016-01-20 13:23:02 -0430:

Greetings,

At the Tokyo summit, we discussed OpenStack's development themes in a
cross-project session. In this session a group of folks started discussing what
topics the overall community could focus on as a shared effort. One of the
things that was raised during this session is the need of having cycles to
stabilize projects. This was brought up by Robert Collins again in a meeting[0]
the TC had right after the summit and no much has been done ever since.

Now, "stabilization Cycles" are easy to dream about but really hard to do and
enforce. Nonetheless, they are still worth a try or, at the very least, a
thought. I'll try to go through some of the issues and benefits a stabilization
cycle could bring but bear in mind that the lists below are not exhaustive. In
fact, I'd love for other folks to chime in and help building a case in favor or
against this.

Negative(?) effects
===

- Project won't get new features for a period of time Economic impact on
  developers(?)
- It was mentioned that some folks receive bonuses for landed features


As uncomfortable as it may be to tell another contributor "no" and
realize that it will have a direct financial impact on them, we
can't encourage a contribution model in which everyone is vying for
their own special patches to land in order to get paid. That destroys
our ability to look at the overall health of the project, and
objectively examine the wisdom of adding (or removing) any given
feature. It moves the issues we have with some vendor-specific
drivers out of the driver space and into all of OpenStack, turning
our open collaboration into competition.


Amen, brother. My thoughts exactly.


Couldn't agree more! (Just to be clear, I mentioned it to be explicit and have a
somewhat objective proposal rather than dropping my opinions here and there :P).

Flavio


-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev