Hi Martin, By collections infrastructure, do you mean something like the collection testers in guava-testlib?
If so, I agree that JUnit 5's dynamic tests look promising for implementing such an infrastructure. I admit I don't have all the context here, but would using guava-testlib in the meantime be a viable medium- or short-term solution? Or would its dependence on JUnit 3/4 make switching impractical? Or, perhaps, has development into CollectionTest gone so far that, for that reason instead, it would be impractical until switch, at least until something superior using e.g. JUnit 5's dynamic tests is made? Kind regards, Jonathan On 19 October 2016 at 23:21, Martin Buchholz <marti...@google.com> wrote: > These tests are maintained outside of openjdk and have only recently been > made available within openjdk. > > There may be a future where all the consumers of these tests are willing to > accept a good test framework. Junit 5 dynamic tests look promising > http://junit.org/junit5/docs/current/user-guide/#writing- > tests-dynamic-tests > but it may be a decade before we can use it. > > Probably code bases that include interfaces (e.g. Collection) should > provide infrastructure to test those interfaces for any implementation, but > openjdk does not try to do so. CollectionTest is an experimental step in > that direction, but it looks we could do better with Junit 5 some day. The > openjdk and jtreg projects can try to help by supporting junit 5 earlier > rather than later. > > On Wed, Oct 19, 2016 at 12:19 PM, Stuart Marks <stuart.ma...@oracle.com> > wrote: > > > > > > > On 10/18/16 11:00 AM, Martin Buchholz wrote: > > > >> There's already a certain amount of mixing, e.g. stream tests sometimes > >> test > >> j.u.c. and Queue tests sometimes test all the Queue implementations. > >> Unlike > >> non-test code, some redundancy and divergence of testing frameworks and > >> styles > >> is fine. I still haven't found a way of writing shared tests for > >> implementations of an interface that I really like. > >> > > > > It's probably the case that divergence of testing frameworks is > > unavoidable, or at least it's impractical. I doubt that it's worthwhile > to > > rewrite all the straight jtreg tests into TestNG, or JUnit, or whatever. > > But I'd draw the line before saying this is "fine." [1] > > > > Maintaining the tests' organization is pretty important. Most of the > > collections tests are in jdk/test/java/util though they're spread around > a > > bit uncomfortably even here. But now it's, oh, ArrayDeque and > > ArrayList.removeIf tests are over *here* instead. Having things spread > > around increases the likelihood of cases being missed or being tested > > redundantly. > > > > The test groups have to be maintained as well. I know I've been bitten by > > problems (not in collections) where I *thought* I had run the right set > of > > tests, but it ended up that I hadn't, resulting in me breaking the build. > > As it turns out, jdk_collections_core doesn't include any ArrayDeque > tests. > > Hmmm. Well, maybe jdk_collections_core isn't useful. It would have been > > nice to know this when I added it last year. :-/ [2] > > > > As things stand, I'm basically OK with this changeset going in. But it > > does seem to highlight some areas that need some future cleanup, > > particularly in the tests and the test group declarations. > > > > s'marks > > > > [1] http://knowyourmeme.com/memes/this-is-fine > > > > [2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/7a0c06013ae6 > > > > >
