On 10/08/2015 02:50 PM, Martin Basti wrote:
On 10/08/2015 02:39 PM, Martin Kosek wrote:
On 10/08/2015 02:08 PM, Oleg Fayans wrote:
Hi,
On 10/08/2015 11:18 AM, Jan Pazdziora wrote:
On Thu, Oct 08, 2015 at 11:12:37AM +0200, Oleg Fayans wrote:
When the ticket is addressed and these workarounds are no longer
needed -- what is our process for finding these workarounds and
reverting them, so that the tests test the real, expected behaviour?
As per discussion with Martin Basti, it was decided that this
workaround
will only be applied to the current 4-2 branch, not to the
upstream. In
That sounds like a reasonable plan for this issue.
upstream the issue itself will (supposedly) be solved
Except currently it's not, so (IIUIC) you will keep having
nondeterministic failures in master.
I was mostly interested in the general approach that we have to
workarounds -- how do we track them, how do we make sure they don't
stick in tests forever, even after the issue was already properly
addressed.
That's actually a great point. I personally would like tickets to
have one more
field: "workaround" containing the address of a workaround in the
format
"path_to_the_file:line_number" or better even - a commit id of this
workaround,
so that once the ticket is resolved, we could easily find what to
reverse.
Please don't add any more trac fields, there is too many of them
already :-)
Keyword may serve better for now...
+1
new trac field for a few workarounds per year is not worth it.
Perhaps we could use pytest's expected fail (xfail) or skip marker. [1]
It would prevent test from failing in the report and once the underlying
issue is fixed, it will raise as an unexpected pass.
It could be used as a temporary solution, once the issue is fixed, we
would remove the mark from the test. This would probably need some
workflow to be defined for these cases.
[1]: https://pytest.org/latest/skipping.html
--
Milan Kubik
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
