On 08/03/18 19:31, Doug Hellmann wrote:
If these new plugins might contain "candidate" tests and all tests
are potentially candidates, how are these new repos different from
the existing repos that already contain all of the tests?
We have to allow candidate tests because otherwise there's a
chicken-and-egg type problem where we can never add tests to an existing
trademark program. But the definition of 'candidate' that I'm thinking
of is along the lines of "under active discussion to be added to the
next iteration of refstack". Perhaps that should be documented.
To answer your question directly, the main way they're different is that
by putting tests in one of these repos, teams are committing to not
really change them, and certainly to ~never break backwards compat.
It seems
like at least part of the problem with the current system was
triggered by confusion about when to move tests around to satisfy
the policy. Can we avoid that problem with the new system? If we're
not going to move the tests into Tempest itself and have the QA
team manage them, why not simply take the tests from the repos where
they already live?
I think a lesson we've learned from Tempest is that there are two parts
to successfully reviewing a change:
1) Determine whether the change affects a test that is part of the
trademark program.
2) Don't make the change if it does.
2 is easy, but 1 is hard. By separating the trademark tests into a
separate repo, we make 1 easy as well. This reduces risk and increases
review throughput.
I thought the QA team no longer wanted to be responsible for these
extra tests. Has that changed again? I've lost track of everyone's
positions, I'm afraid.
Their position as I understand it is:
* They're not going to write tests
* They're happy to document the process, offer advice, and review tests
as time allows
* They don't want tests to be thrown over the transom and made their
problem for the rest of time
That doesn't conflict with anybody's goals here, so it's mostly a matter
of documenting it clearly so that somebody reading without all this
context won't get the wrong idea.
cheers,
Zane.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
