On Mon, Feb 9, 2015 at 2:20 PM, Joe Gordon <joe.gord...@gmail.com> wrote:
> > > On Mon, Feb 9, 2015 at 1:02 PM, John Griffith <john.griffi...@gmail.com> > wrote: > >> On Mon, Feb 9, 2015 at 1:56 PM, Matthew Treinish <mtrein...@kortar.org> >> wrote: >> > On Mon, Feb 09, 2015 at 01:24:34PM -0600, Matt Riedemann wrote: >> >> >> >> >> >> On 2/9/2015 12:23 PM, Joe Gordon wrote: >> >> > >> >> >On Feb 9, 2015 10:04 AM, "Matt Riedemann" <mrie...@linux.vnet.ibm.com >> >> ><mailto:mrie...@linux.vnet.ibm.com>> wrote: >> >> > > >> >> > > There are at least two blocking bugs: >> >> > > >> >> > > 1. https://bugs.launchpad.net/grenade/+bug/1419913 >> >> > > >> >> > > Sounds like jogo is working a javelin fix for this. I'm not aware >> of >> >> >a patch to review though. >> >> > >> >> >We need to stop trying to install tempest in the same env as stable/* >> code. >> >> > >> >> >I should be able to revise/respond to comments shortly. >> >> > >> >> >https://review.openstack.org/#/c/153080/ >> >> > >> >> >https://review.openstack.org/#/c/153702/ >> >> > >> >> >This is also blocking my effort to pin stable dependencies (Dean's >> >> >devstack changes are needed before we can pin stable dependencies as >> well). >> >> > >> >> > > >> >> > > 2. https://bugs.launchpad.net/ceilometer/+bug/1419919 >> >> > > >> >> > > I'm not sure yet what's going on with this one. >> >> > > >> >> >> >> Tracking etherpad: >> >> >> >> https://etherpad.openstack.org/p/wedged-stable-gate-feb-2015 >> > >> > >> > So I think it's time we called the icehouse branch and marked it EOL. We >> > originally conditioned the longer support window on extra people >> stepping >> > forward to keep things working. I believe this latest issue is just the >> latest >> > indication that this hasn't happened. Issue 1 listed above is being >> caused by >> > the icehouse branch during upgrades. The fact that a stable release was >> pushed >> > at the same time things were wedged on the juno branch is just the >> latest >> > evidence to me that things aren't being maintained as they should be. >> Looking at >> > the #openstack-qa irc log from today or the etherpad about trying to >> sort this >> > issue should be an indication that no one has stepped up to help with >> the >> > maintenance and it shows given the poor state of the branch. >> > >> > If I'm not mistaken with our original support window lengths Icehouse >> would be >> > EOL'd around now. So it's time we stopped pretending we'll be >> maintaining this >> > branch for several more months and just go through the normal EOL >> procedure. >> > >> > >> Was this serious? I mean, we just say; 'sorry, yes we said support >> until X; but now it's hard so we're going to drop it'. >> >> Tell me I'm missing something here? >> > > You are missing the fact that a bunch of us (Matt Treinish, myself and > others) are frustrated by the fact that we end up fixing stable branches > whenever they break because we touch tempest, grenade and other projects > that require working stable branches. But we do not want to be working on > stable branches ourselves. I begrudgingly stepped up to work on pinning > all requirements on stable branches, to reduce the number of times stable > branches break and ruin my day. But my plan to cap dependencies has been > delayed several times by stable branches breaking again and again, along > with unwinding undesired behaviors in our testing harnesses. > Note: At least 3 of us just spent most of the day working on this instead of working on developing on other things. > > Most recently, stable/juno grenade broke on February 4th (due to the > release of tempest-lib 0.2.0). This caused bug > https://bugs.launchpad.net/grenade/+bug/1419913 > <https://bugs.launchpad.net/grenade/+bug/1419913> > (" pkg_resources.ContextualVersionConflict: (oslo.config 1.4.0 > (/usr/local/lib/python2.7/dist-packages), > Requirement.parse('oslo.config>=1.6.0'), set(['tempest-lib']))". This > specific bug is caused because we install master tempest (due to branchless > tempest) on stable/icehouse and sync in stable/icehouse global requirements > which not surprisingly has a conflict with tempest's requirements. So the > solution here is stop installing tempest and requiring it to work with > stable/icehouse, stable/juno and master's version of global-requirements. > But that doesn't work because master tempest has an uncapped version of > boto but nova stable/icehouse only works with the capped version of > Icehouse. So we get this https://review.openstack.org/#/c/154217/1/. So > now we are exploring dropping the EC2 tests on stable/icehouse. If that > works, we still need to land roughly 4 more patches to unwedge this > stable/juno grenade and prevent this type of issue from happening in the > future. > > Lets say we EOL Icehouse, we stop running grenade on stable/juno patches. > Meaning this bug goes away all together and stable/juno is unwedged and I > can move forward with pinning all stable/juno requirements. > > What I expect to happen when issues like this arise is interested parties > work together to fix things and be proactive and make stable testing more > robust. Instead we currently have people who have no desire to work on > stable branches maintaining them. Pinning all direct stable/* requirements > isn't enough to make sure stable/* doesn't break. There are transitive > dependencies that can change (I have a plan on how to pin those too, but it > will take time and I can use some help), and changing packages etc. can > break things as well. Having a reactive stable maintenance team is > insufficient. > > Until we have a proactive team of people actually maintaining stable > branches and related testing infrastructure, versus the skeleton crew we > have now, I don't think we should support stable branches for 15 months. > Because supporting Icehouse for 15 months means we will have to support 3 > stable branches 50% of the time. Currently the plan is to EOL Icehouse 4 > months into the M release, so we will need to support Icehouse, Juno, Kilo > and Master. And considering the issues we are having with supporting two > stable branches, supporting 3 scares me. > > > > >> > -Matt Treinish >> > >> > >> __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
