On 5/7/2012 2:29 PM, Venkatesan, Ravikumar wrote:
On 5/4/2012 9:08 AM, Jay Pipes wrote:
On 05/04/2012 06:14 AM, Karajgi, Rohit wrote:
Hi,
What is the policy that we should or are following for test cases that
fail due to an existing Open Bug in Launchpad?
For eg:
https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_floating_ips.py#L64
skips the test and posts the Bug ID in the message.
Do we submit the test for review with the @skip(until Bug #xyz is fixed)
decorator applied?
This is the choice that I believe is the easiest and simplest. We just
need to be vigilant to remove the skips when/if the bug is fixed.
-jay
Jay, I agree with the idea that we should check the test in anyway. But
we have the issue of who remembers to remove the skip decorator when
the bug is fixed. Since we are going to be running tempest nightly, I
think it might be better to make these tests fail in the absence of the
bug rather than being skipped. I think there may already be some cases
in Tempest that do this.
-David
I agree with David, and I prefer that the test failed rather than using skip
decorator.
- Ravi
Ravi, after hearing Jays comments I realized that we can't do this if we
are going to use tempest to gate trunk checkins for other projects. If
we make the test fail, then it will be impossible for a developer to
check in a fix to a bug unless they also change the tempest test. In an
ideal world this would be the case, as surely they already do this for
unit tests, but I don't think we are ready for that. I thought I sent a
message to that effect last week but perhaps it never went through.
-David
--
Mailing list: https://launchpad.net/~openstack-qa-team
Post to : openstack-qa-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack-qa-team
More help : https://help.launchpad.net/ListHelp
