On 10/02/16 18:48, "Clint Byrum" <cl...@fewbar.com> wrote:
>Excerpts from Sean Dague's message of 2016-02-10 04:33:44 -0800: >> The largeops tests at this point are mostly finding out that some of our >> new cloud providers are slow - http://tinyurl.com/j5u4nf5 >> >> This is fundamentally a performance test, with timings having been tuned >> to pass 98% of the time on two clouds that were very predictable in >> performance. We're now running on 4 clouds, and the variance between >> them all, and between every run on each can be as much as a factor of 2. >> >> We could just bump all the timeouts again, but that's basically the same >> thing as dropping them. >> >> These tests are not instrumented in a way that any real solution can be >> addressed in most cases. Tests without a path forward, that are failing >> good patches a lot, are very much the kind of thing we should remove >> from the system. >> > >I think we need to replace this with something that measures work >counters, and not clock time. As you say, some of the other test suites >out there already pick up a lot of this slack too. Also, I'm working at >this as well with the counter-inspection spec, so hopefully dropping >this now won't leave too much of a gap in coverage while we ramp up >counter-inspection. > >+1 to getting rid of it now, as instability in the test suites, which >slows down development velocity, is worse than missing a few performance >regressions in the corners. I believe there is now a performance working group as part of the large deployments team (https://wiki.openstack.org/wiki/Performance_Team). Have they been contacted to determine the alternative scenarios and how to strengthen the testing ? Tim > >__________________________________________________________________________ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
smime.p7s
Description: S/MIME cryptographic signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
