[
https://issues.apache.org/jira/browse/CASSANDRA-12277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397228#comment-15397228
]
Branimir Lambov commented on CASSANDRA-12277:
---------------------------------------------
After trying for a day to find a quick solution to the underlying problem with
this test, i.e. to find a way to algorithmically generate a good selection for
the tokens of the first RF nodes, the thing I understand is that the problem
needs a non-trivial time investment.
Being also too lazy to keep arguing about the benefits of an easy-to-use
flaking test rerun infrastructure, I whipped out a rerun method that is not too
difficult to apply on a case by case basis, and wrapped the test in it. The
code is here:
|[code|https://github.com/blambov/cassandra/tree/12277]|[utests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-12277-testall/]|[dtests|http://cassci.datastax.com/view/Dev/view/blambov/job/blambov-12277-dtest/]|
The {{flakyTest}} method runs the given test, and if it fails reruns it for the
specified number of times expecting it to succeed on all reruns. This should
reduce the incidence of flakes on this test by at least an order of magnitude
while immediately breaking if the test fails with higher than expected
probability.
> Extend testing infrastructure to handle expected intermittent flaky tests -
> see ReplicationAwareTokenAllocatorTest.testNewCluster
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-12277
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12277
> Project: Cassandra
> Issue Type: Bug
> Reporter: Joshua McKenzie
> Assignee: Branimir Lambov
> Priority: Minor
> Labels: test
>
> From an offline discussion:
> bq. The ReplicationAwareTokenAllocatorTest.testNewCluster failure is a flake
> -- randomness will sometimes (on the order of 1/100) cause it to fail.
> Extending the ranges to avoid these flakes goes too far and makes the test
> meaningless.
> bq. How about instead of @flaky/@Ignore which currently indicates a test that
> intermittently fails but we do not expect it to, we instead use @tries, or
> @runs, or some annotation that indicates "run this thing N times, if M pass
> we're good". This would allow us to keep the current "we don't care about
> these test results (in context of green test board) because intermittent
> failures are not expected and the test quality needs shoring up" from "we
> expect this test to fail sometimes in this particular way."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
