Good post Jiri, this got me thinking :) Running the entire testsuite again with uber jars would add a lot of time to the build time.
Maybe we should have a set of tests that must be executed for sure, e.g. like Wildfly's smoke tests [1]. We have "functional" group but right now it covers pretty much all tests. Such tests should live in a separate testsuite, so that we could add the essential tests for *all* components. In a way, we've already done some of this in integrationtests/ but it's not really well structured for this aim. Also, if we would go down this path, something we should take advantage of (if possible with JUnit/TestNG) is what Gustavo did with the Spark tests in [2], where he used suites to make it faster to run things, by starting a cache manager for distributed caches, running all distributed tests...etc. In a way, I think we can already do this with Arquillian Infinispan integration, so Arquillian would probably well suited for such smoke testsuite. Thoughts? Cheers, [1] https://github.com/wildfly/wildfly#running-the-testsuite [2] https://github.com/infinispan/infinispan-spark/tree/master/src/test/scala/org/infinispan/spark -- Galder Zamarreño Infinispan, Red Hat ----- Original Message ----- > Hi Jiri, comments inline. > > On 2.9.2015 10:40, Jiri Holusa wrote: > > Hi all, > > > > we've been thinking for a while, how to test ISPN uber jars. The current > > status is that we actually don't have many tests in the testsuite, there > > are few tests in integrationtests/all-embedded-* modules that are > > basically copies of the actual tests in corresponding modules. We think > > that this test coverage is not enough and more importantly, they are > > duplicates. > > > > The questions are now following: > > * which tests should be invoked with uber-jars? Whole ISPN testsuite? Only > > integrationtests module? > > The goal is to run the whole test suite because, as you said, we don't > have enough tests in integrationtests/* And we can't duplicate all > test classes from individual modules here. > > > * how would it run? Create Maven different profiles for "classic" jars and > > uber jars? Or try to use some Maven exclusion magic if even possible? > > > > Some time ago, we had discussion about this with Sebastian, who suggested > > that running only integrationtests module would be sufficient, because > > uber-jars are really about packaging, not the functionality itself. But I > > don't know if the tests coverage is sufficient in that level, I would be > > much more confident if we could run the whole ISPN testsuite against > > uber-jars. > Right. Uber-jars are about packaging but you don't know that the > packiging is right until you try all the features and see that > everything works. There might be some classes missing (just for some > particular features), same classes in different packages, the > Manifest.mf might be corrupted and then something won't work in OSGi. > > > I'd prefer a separate Maven profile. IMO, exclusions are too error-prone. > > > Martin > > > > I'm opening this for wider discussion as we should agree on the way how to > > do it, so we could do it right :) > > > > Cheers, > > Jiri > > > > > > _______________________________________________ > infinispan-dev mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list [email protected] https://lists.jboss.org/mailman/listinfo/infinispan-dev
