Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
I had the labels wrong - here's a slightly better link - http://bit.ly/1gdxYeg On Fri, Dec 6, 2013 at 4:31 PM, Davanum Srinivas wrote: > Joe, > > Looks like we may be a bit more stable now? > > Short URL: http://bit.ly/18qq4q2 > > Long URL : > http://graphite.openstack.org/graphlot/?from=-120hou

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Davanum Srinivas
Joe, Looks like we may be a bit more stable now? Short URL: http://bit.ly/18qq4q2 Long URL : http://graphite.openstack.org/graphlot/?from=-120hour&until=-0hour&target=color(alias(movingAverage(asPercent(stats.zuul.pipeline.gate.job.gate-tempest-dsvm-full.SUCCESS,sum(stats.zuul.pipeline.gate.job

Re: [openstack-dev] Top Gate Bugs

2013-12-06 Thread Matt Riedemann
On Wednesday, December 04, 2013 7:22:23 AM, Joe Gordon wrote: TL;DR: Gate is failing 23% of the time due to bugs in nova, neutron and tempest. We need help fixing these bugs. Hi All, Before going any further we have a bug that is affecting gate and stable, so its getting top priority here. e

[openstack-dev] Top Gate Bugs

2013-12-04 Thread Joe Gordon
TL;DR: Gate is failing 23% of the time due to bugs in nova, neutron and tempest. We need help fixing these bugs. Hi All, Before going any further we have a bug that is affecting gate and stable, so its getting top priority here. elastic-recheck currently doesn't track unit tests because we don't

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Christopher Yeoh
On Fri, Nov 22, 2013 at 2:28 AM, Matt Riedemann wrote: > > > On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: > >> On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi >> wrote: >> >>> Hi Joe, >>> >>> 2013/11/20 Joe Gordon : >>> Hi All, As many of you have noticed the

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Ken'ichi Ohmichi
Hi Clark, 2013/11/21 Clark Boylan : > > Joe seemed to be on the same track with > https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:57578,n,z > but went far enough to revert the change that introduced that test. A > couple people were going to keep hitting

Re: [openstack-dev] Top Gate Bugs

2013-11-21 Thread Matt Riedemann
On Wednesday, November 20, 2013 11:53:45 PM, Clark Boylan wrote: On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi wrote: Hi Joe, 2013/11/20 Joe Gordon : Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Clark Boylan
On Wed, Nov 20, 2013 at 9:43 PM, Ken'ichi Ohmichi wrote: > Hi Joe, > > 2013/11/20 Joe Gordon : >> Hi All, >> >> As many of you have noticed the gate has been in very bad shape over the >> past few days. Here is a list of some of the top open bugs (without pending >> patches, and many recent hits)

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Ken'ichi Ohmichi
Hi Joe, 2013/11/20 Joe Gordon : > Hi All, > > As many of you have noticed the gate has been in very bad shape over the > past few days. Here is a list of some of the top open bugs (without pending > patches, and many recent hits) that we are hitting. Gate won't be stable, > and it will be hard t

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Robert Collins
We settled on 1251920. https://review.openstack.org/57509 is the fix for that bug. Note that Oslo was fixed on Jun 28th, nova hasn't synced since then. If we were using oslo as a library we would have had the fix as soon as olso did a release. These are the references to strong_store - and thus

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Robert Collins
Which of these bugs would be appropriate to use for the fix to strong_store - it affects lockutils and rpc, both of which are going to create havoc :) -Rob On 21 November 2013 07:19, Salvatore Orlando wrote: > I've noticed that > https://github.com/openstack/nova/commit/85332012dede96fa6729026c2

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Robert Collins
I'm putting up a patch now. On 21 November 2013 07:19, Salvatore Orlando wrote: > I've noticed that > https://github.com/openstack/nova/commit/85332012dede96fa6729026c2a90594ea0502ac5 > stores the network client in local.strong_store which is a reference to > corolocal.local (the class, not the i

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Salvatore Orlando
I've noticed that https://github.com/openstack/nova/commit/85332012dede96fa6729026c2a90594ea0502ac5stores the network client in local.strong_store which is a reference to corolocal.local (the class, not the instance). In Russell's example instead the code accesses local.store which is an instance

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Russell Bryant
On 11/20/2013 12:21 PM, Alex Gaynor wrote: > Nope, you're totally right, corolocal.local is a class, whose instances > are the actual coroutine local storage. But I don't think his example is what is being used. Here is an example using the openstack.common.local module, which is what nova uses f

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Alex Gaynor
Nope, you're totally right, corolocal.local is a class, whose instances are the actual coroutine local storage. Alex On Wed, Nov 20, 2013 at 9:11 AM, Roman Podoliaka wrote: > Hey all, > > I think I found a serious bug in our usage of eventlet thread local > storage. Please check out this snippe

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Roman Podoliaka
Hey all, I think I found a serious bug in our usage of eventlet thread local storage. Please check out this snippet [1]. This is how we use eventlet TLS in Nova and common Oslo code [2]. This could explain how [3] actually breaks TripleO devtest story and our gates. Am I right? Or I am missing s

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Derek Higgins
On 20/11/13 14:21, Anita Kuno wrote: > Thanks for posting this, Joe. It really helps to create focus so we can > address these bugs. > > We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. > > We are looking for someone to work on 1251784 - I had mentioned it at > Monday's

Re: [openstack-dev] Top Gate Bugs

2013-11-20 Thread Anita Kuno
Thanks for posting this, Joe. It really helps to create focus so we can address these bugs. We are chatting in #openstack-neutron about 1251784, 1249065, and 1251448. We are looking for someone to work on 1251784 - I had mentioned it at Monday's Neutron team meeting and am trying to shop it aroun

[openstack-dev] Top Gate Bugs

2013-11-19 Thread Joe Gordon
Hi All, As many of you have noticed the gate has been in very bad shape over the past few days. Here is a list of some of the top open bugs (without pending patches, and many recent hits) that we are hitting. Gate won't be stable, and it will be hard to get your code merged, until we fix these b