On 10/09/2015 11:03 AM, Jan Pazdziora wrote:
On Fri, Oct 09, 2015 at 10:31:32AM +0200, Tomas Babej wrote:
a heavy process. Also, I wouldn't be strict about it, as we already have
a couple of workarounds, and not every time a workaround has a exact
mapping to a particular ticket.
Frankly, this worries me much more than not having ticket for some
trivial change to the code base.
If there's workaround in tests, it's some action that we do but
users/admins don't in their real setups. And that can cause failures
for our users that we don't see or even no longer know about because
in our tests, we've cleverly worked around them.
I guess, the global question of whether to do workarounds in tests to
make them pass or not is older than the very oldest profession on earth.
I must admit, I am on Jan's side here. I would prefer to implement the
approach proposed by Milan: mark the test scenario as expected failure
(if it is crucial to make the whole run pass), or better even to leave
it as it is: just so that each red CI run would remind of the necessity
to fix the original issue.
This all is a theory, however. What do we do with this particular case?
It's an edge case (does anyone except root zone admins sign a root
zone?). Should we probably disable the whole scenario? Or just leave it
failing as it is?
Either that workaround step is needed and needs to be documented, or
that step should not be needed and there should be a ticket describing
the issue.
--
Oleg Fayans
Quality Engineer
FreeIPA team
RedHat.
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
