just to be clear, as i mentioned: tests.iters should run setup/teardown just fine, thats not a problem. There is no issue with tests.iters setup/teardown/Before/After.
but its only going to instantiate your *class* a single time and run *class*-level stuff (e.g. BeforeClass/AfterClass) a single time. On Mon, Jan 15, 2018 at 11:35 AM, Erick Erickson <erickerick...@gmail.com> wrote: > Any test that doesn't really do anything in setup/teardown seems like a > good candidate for using tests.iters, which will be much faster. > > For SolrCloud level tests beasting is becoming my first choice. > > For a lot of the Lucene level code tests.iters makes a lot of sense so > it's too useful to remove. > > Best, > Erick > > > On Mon, Jan 15, 2018 at 7:56 AM, Jason Gerlowski <gerlowsk...@gmail.com> > wrote: >> It sounds like the recommendation in this thread is to _always_ use >> "ant beast" instead of "tests.iters". Is there _any_ case where >> "tests.iters" should be preferred? If not, should we remove support >> for "tests.iters" to remove any ambiguity? (Especially since this has >> come up on the list a few times...) >> >> On Fri, Jan 12, 2018 at 11:14 AM, Erick Erickson >> <erickerick...@gmail.com> wrote: >>> Well, since I'm in there anyway I'll include the note in the patch. At >>> least that'll alert people to dig deeper. >>> >>> On Thu, Jan 11, 2018 at 8:34 PM, David Smiley <david.w.smi...@gmail.com> >>> wrote: >>>> Yeah thanks guys -- beast it is. >>>> >>>> I wonder if we should not document tests.iters (a bit more expert), or add >>>> a >>>> warning to it in the help output saying something like: NOTE: some tests >>>> are >>>> incompatible because BeforeClass/AfterClass isn't performed inbetween. Try >>>> beast.iters instead. >>>> >>>> On Thu, Jan 11, 2018 at 8:39 PM Erick Erickson <erickerick...@gmail.com> >>>> wrote: >>>>> >>>>> Ok, thanks both. That makes a lot of sense. I'll just use beasting for >>>>> most anything SolrCloud related. >>>>> >>>>> >>>>> On Thu, Jan 11, 2018 at 4:56 PM, Chris Hostetter >>>>> <hossman_luc...@fucit.org> wrote: >>>>>> >>>>>> : (I had left the comment in question) >>>>>> : I think a test shouldn't have to explicitly clean up after itself, >>>>>> except >>>>>> : perhaps intra-method as-needed; test-infrastructure should do the class >>>>>> : (test suite). >>>>>> >>>>>> All test code should always be expected to clean up their messes at >>>>>> whatever "ending" stage corrisponds with the stage where the mess was >>>>>> made. >>>>>> >>>>>> how the mess is cleaned up, and wether infrastructure/scaffolding code >>>>>> helps do that dpeends on the specifics of the infrastucture/scaffolding >>>>>> in >>>>>> question -- if you make a mess in a test method that the general purpose >>>>>> infrastructure doesn't expect, then the burden is on you >>>>>> to add the level of infrastructure (either in your specific test class, >>>>>> or >>>>>> in a new abstract base class depending on how you think it might be >>>>>> re-usable) to do so. >>>>>> >>>>>> In the abstract: Assume AbstractParentTest class that creates some >>>>>> "parentMess" in @BeforeClass, and deletes "parentMess" in an >>>>>> @AfterClass.... >>>>>> >>>>>> 1) if you want all of your tests methods to have access to a >>>>>> shiny new/unique instance of "childMess" in every test method, then >>>>>> burden >>>>>> is on you to create/destroy childMess in your own @Before and @After >>>>>> methods >>>>>> >>>>>> 2) If you want test methods that are going to mutate "parentMess" then >>>>>> the >>>>>> burden is on you to ensure (ideally via @After methods that "do the right >>>>>> thing" even if the test method fails) that "parentMess" is correctly >>>>>> reset >>>>>> so that all the test methods depending on "parentMess" can run in any >>>>>> order (or run multiple times in a single instance) ... either that, or >>>>>> you >>>>>> shouldn't use AbstractParentTest -- you should create/destroy >>>>>> a "parentMess" instance yourself in your @Before & @After methods >>>>>> >>>>>> Concretely... >>>>>> >>>>>> : > The assumption was that everything would be cleaned up between runs >>>>>> : > doesn't appear to be true for SolrCloud tests. I think one of two >>>>>> things is >>>>>> : > happening: >>>>>> : > >>>>>> : > 1> collections (and perhaps aliases) are simply not cleaned up >>>>>> : > >>>>>> : > 2> there is a timing issue, we have waitForCollectionToDisappear in >>>>>> test >>>>>> : > code after all. >>>>>> >>>>>> ...these are vague statements ("everything", "SolrCloud tests", "not >>>>>> cleaned up") and not being intimately familiar with the test class in >>>>>> question it's not clear exactly is happening or what expectations various >>>>>> people have -- BUT -- assuming this is in regards to >>>>>> SolrCloudTestCase, that base class has very explicit docs about >>>>>> how it's intended to be used: you are expected to configure & init a >>>>>> MiniSolrCloudCluster instance in an @BeforeClass method -- it has helper >>>>>> code for this -- and that cluster lives for the lifespan of the class at >>>>>> which point an @AfterClass in SolrCloudTestCase will ensure it gets torn >>>>>> down. >>>>>> >>>>>> Tests which subclass SolrCloudTestCase should be initializing the cluster >>>>>> only in @BeforeClass. Most tests should only be creating collections in >>>>>> @BeforeClass -- allthough you are certainly free to do things like >>>>>> create/destroy collections on a per test method basis in @Before/@After >>>>>> methods if you have a need for that sort of thing. >>>>>> >>>>>> If that's not the lifecycle you want -- if you want a lifecycle where >>>>>> ever >>>>>> individual test method gets it's own pristine new MiniSolrCloudCluster >>>>>> instance w/o any pre-existing collections, then you shouldn't use >>>>>> SolrCloudTestCase -- you should just create/destroy >>>>>> unique MiniSolrCloudCluster instances in your own @Before/@After methods. >>>>>> >>>>>> >>>>>> Bottom Line: there is no one size fits all test scaffolding -- not when >>>>>> we >>>>>> have some tests classes where we want to create a collection once, fill >>>>>> it >>>>>> with lots of docs, and then re-use it in 100s of test methods, but other >>>>>> classes want to test the very operation of creating/deleting collections. >>>>>> >>>>>> Use the tools that make sense for the test you're writting. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -Hoss >>>>>> http://www.lucidworks.com/ >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >>>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >>>> http://www.solrenterprisesearchserver.com >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org