I agree with deleting @Ignored tests except for the few that have JIRA
tickets open for them. There are less than 1/2 dozen of these and we
should consider keeping them since we have a way of tracking them.
On 12/31/19 2:07 PM, Alexander Murmann wrote:
Here are a few things that are true for me or I believe are true in general:
- Our test suite is more flaky than we'd like it to be
- I don't believe that adding more Unit tests that follow existing
patterns buys us that much. I'd rather see something similar to what some
folks are doing with Membership right now where we isolate the code and
test it more systematically
- We have other testing gaps: We have benchmarks 👏🎉, but we are still
lacking coverage in that ares; our community is still lacking HA tests. I'd
rather fill those than bring back old DUnit tests that are chosen somewhat
at random.
- I'd rather be deliberate about what tests we introduce than wholesale
bring back a set of tests, since any of these re-introduced tests has a
potential to be flaky. Let's make sure our tests carry their weight.
- If we delete these tests, we can always go back to a SHA from today
and bring them back at a later date
- These tests have been ignored since a very long time and we've shipped
without them and nobody has missed them enough to bring them back.
Given all the above, my vote is for less noise in our code, which means
deleting all ignored tests. If we want to keep them, I'd love to hear a
plan of action on how we bring them back. Having a bunch of dead code helps
nobody.
On Tue, Dec 31, 2019 at 1:50 PM Mark Hanson <mhan...@pivotal.io> wrote:
Hi All,
As part of what I am doing to fix flaky tests, I periodically come across
tests that are @Ignore’d. I am curious what we would like to do with them
generally speaking. We could fix them, which would seem obvious, but we are
struggling to fix flaky tests as it is. We could delete them, but those
tests were written for a reason (I hope). Or we could leave them. This
pollutes searches etc as inactive code requiring upkeep at least.
I don’t have an easy answer. Some have suggested deleting them. I tend to
lean that direction, but I thought I would consult the community for a
broader perspective.
Thanks,
Mark
