[
https://issues.apache.org/jira/browse/SOLR-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457344#comment-13457344
]
Yonik Seeley commented on SOLR-3766:
------------------------------------
Some concerns:
- These tests should not be excluded from the normal non-jenkins command line
"ant test"
- Some tests may only be unstable on our freebsd instance (because of
undesirable configuration like blackhole). I would not want to see such tests
excluded from failing the build on all jenkins/automated tests. This goes back
to Mark's request for a dont-run-on-our-freebsd annotation (which makes more
sense now that we have other jenkins servers running).
- Some tests may only be unstable for certain JVMs
There should also be consensus before a test gets thrown into a place where
people won't notice (or care about) it's failure. Without that, we'd need to
come to consensus on an unambiguous policy about when a test may be marked such
that it will no longer fail builds.
Maybe an easier solution would still be to send mail on failure, and then
people who were not interested could exclude it in a mail filter.
> A test group and build plan for known unstable tests
> ----------------------------------------------------
>
> Key: SOLR-3766
> URL: https://issues.apache.org/jira/browse/SOLR-3766
> Project: Solr
> Issue Type: Task
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
>
> As discussed on the mailing list and suggested by Hoss:
> {code}
> 1) slap a special "@UnstableTest" annotation on them
> 2) set up a new jenkins job that *only* runs these @UnstableTest jobs
> 3) configure this new jenkins job to not send any email
> {code}
> Point 1 is trivial, 2 I have filed an issue for in the runner --
> https://github.com/carrotsearch/randomizedtesting/issues/122
> that I will take care of shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
