We found that random failures in tests caused the team to ignore test failures until there were two or more failures in a row. Unfortunately, that meant they were delaying fixing the problems until there were several cycles of failures, which tended to compound the failures.
Ultimately we decided that tests which failed randomly needed to either be made consistent or removed from the "main line" test suite. That allowed the team to trust that any break in the "main line" test suite needed immediate attention, and breaks in the "unreliable" test suite could be handled more casually. I would think that filtering exceptions (or test failures) may cause bad behavior in your team, since they will be mislead into thinking the tests confirm things are OK, when the random test failures may be pointing to a real bug. Mark Waite On Fri, Aug 8, 2014 at 9:05 AM, Albert Tresens <albert.tres...@gmail.com> wrote: > Hi, > > I am trying to optimize the triaging time on jenkins failues caused by > false positives. There is a percentage of failures that are always self > healed after subsequent builds. Mostly dependent on the underlying OS or > some resource factors. > > Does someone followed any specific approach to address such a situations?. > I guess is a common problem. > > I thought about spawning the Jenkins jobs so I get duplicated results and > discard if its not a double failure or adding plugins for filtering > specific exceptions. > > Any suggestion or alternatives? > > Thanks! > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Thanks! Mark Waite -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
