[
https://issues.apache.org/jira/browse/LUCENE-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122068#comment-13122068
]
Dawid Weiss edited comment on LUCENE-3492 at 10/6/11 5:04 PM:
--------------------------------------------------------------
Hi Shai. This is definitely not only for debugging. For example we use
randomized testing inside CarrotSearch to test algorithmic/ combinatorial code.
Once you hit a bug, you simply copy the test case (or a call to a common test
case method) and fix the seed to have a regression test for the future (so that
you know you're not failing examples that previously failed). So, for example:
{code}
@Test @Seed("23095324")
public void runFixedRegression_1 { doSomethingWithRandoms(); }
@Test @Seed("239735923")
public void runFixedRegression_2 { doSomethingWithRandoms(); }
@Test
public void runRandomized { doSomethingWithRandoms(); }
{code}
This is a scenario I really came to like. It's a bit like your tests write
themselves for you :)
I left system properties for fixing seeds and enforcing repetition number
because they are currently in Lucene, although I personally don't like them
that much (because they affect everything globally). I do understand they're
useful for quick hacking without recompiling stuff or for remote executions,
but I'd much rather have something like -Dseed.testClass[.method]=xxxx which
would affect only a single class or method rather than everything. The same can
be done for filtering which method/ test case to execute. This is debatable of
course and a matter of personal taste.
I should publish what I have tonight on github (I'm moving certain things out
of our proprietary codebase and there are JUnit corner cases that slow things
down).
was (Author: dweiss):
Hi Shai. This is definitely not only for debugging. For example we use
randomized testing inside CarrotSearch to test algorithmic/ combinatorial code.
Once you hit a bug, you simply copy the test case (or a call to a common test
case method) and fix the seed to have a regression test for the future (so that
you know you're not failing examples that previously failed). So, for example:
{code}
@Test @Seed("23095324")
public void runFixedRegression_1 { doSomethingWithRandoms(); }
@Test @Seed("239735923")
public void runFixedRegression_1 { doSomethingWithRandoms(); }
@Test
public void runRandomized { doSomethingWithRandoms(); }
{code}
This is a scenario I really came to like. It's a bit like your tests write
themselves for you :)
I left system properties for fixing seeds and enforcing repetition number
because they are currently in Lucene, although I personally don't like them
that much (because they affect everything globally). I do understand they're
useful for quick hacking without recompiling stuff or for remote executions,
but I'd much rather have something like -Dseed.testClass[.method]=xxxx which
would affect only a single class or method rather than everything. The same can
be done for filtering which method/ test case to execute. This is debatable of
course and a matter of personal taste.
I should publish what I have tonight on github (I'm moving certain things out
of our proprietary codebase and there are JUnit corner cases that slow things
down).
> Extract a generic framework for running randomized tests.
> ---------------------------------------------------------
>
> Key: LUCENE-3492
> URL: https://issues.apache.org/jira/browse/LUCENE-3492
> Project: Lucene - Java
> Issue Type: Improvement
> Components: general/test
> Reporter: Dawid Weiss
> Assignee: Dawid Weiss
> Priority: Minor
> Fix For: 4.0
>
> Attachments: Screen Shot 2011-10-06 at 12.58.02 PM.png
>
>
> I love the idea of randomized testing. Everyone (we at CarrotSearch, Lucene
> and Solr folks) have their glue to make it possible. The question is if
> there's something to pull out that others could share without having the need
> to import Lucene-specific classes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
