|
||||||||
|
This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira |
||||||||
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
In our case, we have several thousand functional tests and an engineering organization of roughly 150 people that contribute to them. That said, we ocasionally have a an intermittent failure or two that we'd like to deal with in isolation, but we don't want the build to fail for hours while the fix is on its way. We're okay with tolerating roughly 1% of failures, but really nothing more.
Obviously, this is an opt-in feature, so I'm not sure how it could do anything but benefit others who have a similar use case as us. Fundamentally I agree with you that we might not want to ignore a certain percentage of failures, but each organization is different, so shouldn't we let them make that decision?
This feature works similarly to the xUnit-plugin and we could use that instead, but it currently doesn't have TestNG support. We'd prefer to continue to use the TestNG plugin instead of extending xUnit to support TestNG.
Finally, I think that the health extension a la emma is good, but it's not as valuable for our purposes as success vs. unstable vs. failure.
Thanks for responding. Before I move forward, I'll let Nikolay share his experiences too (he originally filed the feature request).