Hi Rob,
Thanks a lot for your post.
Your use cases are very illustrative. I am just starting to tackle the
situation and would like to come up with a solution that can at least
reduce the time I spent triaging and debugging false positives.
The failures are not deterministic so is not that a
Ugh, intermittent test failures. I lived in that jungle for years. In a
previous life, my company used Jenkins to drive a complete homebrew solution.
We wrote a tool to parse the log and write to database, wrote a webapp to read
the database and let you know what failed, even wrote a tool to
We found that random failures in tests caused the team to ignore test
failures until there were two or more failures in a row. Unfortunately,
that meant they were delaying fixing the problems until there were several
cycles of failures, which tended to compound the failures.
Ultimately we decided
