On Wed, Feb 17, 2016 at 06:58:18PM -0500, Sean Dague wrote: > On 02/17/2016 05:05 PM, Doug Hellmann wrote: > > Excerpts from Tony Breeds's message of 2016-02-18 08:04:00 +1100: > <snip> > >> I think we're in a tough spot. > >> > >> My $0.02 is that we have to cap at <0.18.0 however .... > >> > >> We're (the openstack community) finding issues with eventlet which is good > >> (painful I agree, but good) for both communities. However we *are* trying > >> to > >> lock down our release, and test/gate failures are very unwelcome in this > >> phase. > >> > >> 0.18.2 was "mostly" good but shows the following bug: > >> https://bugs.launchpad.net/nova/+bug/1544801 > >> > >> The eventlet team have been very responsive to our issue reports. I'm > >> worried > >> if we cap at 0.18.2 (or 0.17.4) the underlying problems will go undressed > >> and > >> unnoticed for too long and then we'll all loose :( Not least of all > >> because > >> it's *very* likely if we cap this we'll release with the cap which means > >> we're > >> basically stuck with it for the whole life-cycle of mitaka[1][2] > > > > The current plan is to not cap, but exclude the bad versions. The patch > > to do that (https://review.openstack.org/#/c/281479/) does set the > > constraint to 0.18.2, so if we need to change that please drop a comment > > on the review. > > > >> Calls for "extra vetting" of new eventlet releases are slightly > >> problematic, > >> and really require a short cross-project team to verify, as the "Yup is > >> passes > >> dsvm-tempest-full" doesn't cut it. > > > > Can we construct a better automated test scenario? > > Question. Are we only tripping this up in unit tests because the tests > are doing things we'd never really do in real life?
In the case of (nova): https://bugs.launchpad.net/nova/+bug/1544801 https://github.com/eventlet/eventlet/issues/299 It was seen in a "in the wild". That particular issue didn't seem to break anything (for nova) but did record lots of tracebacks in logs. The keystone failure with 0.18.2 *was* limited to tests and has been worked around for the time being. https://github.com/eventlet/eventlet/issues/296 The Neutron issue: https://bugs.launchpad.net/neutron/+bug/1546506 https://github.com/eventlet/eventlet/issues/301 I don't think this was a test-only scenario, and was a clear regression between 0.18.3 and 0.18.3 In my opinion we'll need something to increase faith in a new release, be that manual effort or better tests/tools The challenge seems to be getting back to a point where we can function. I suggested moving back to 0.17.4, but that's going to be hard[1] as we've made several library releases that have "evelntlet>=0.18.2" So perhaps [2] is the right thing to do, as it was mostly good. I think we can deal with the log spam while we test 0.18.4 Like I said earlier the eventlet team has been very responsive and I'm certainly not ignoring that. We just need to be confident that whichever approach we take we can complete it my R-5 > For instance, the nova bug was because someone decided to test the > eventlet wsgi server with a sample app with a custom written eventlet > http client that was doing raw socket reads naively. Which was never a > good idea. The fix: use requests as our http client. Life was fine. Yeah, there have certainly been some test-only issues. > I think it's fine if we're going to want some better tests for bumping > upper-constraints.txt, even if it's tests we only run for certain touchy > packages like eventlet. However we should make sure it wasn't just our > tests being kind of bonkers. I *think* we've passed the point where we're seeing issues restricted to the unit/functional/tempest tests. Yours Tony. [1] https://review.openstack.org/#/c/281479/2 [2] https://review.openstack.org/#/c/281479/1
signature.asc
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
