> On 02 Sep 2016, at 10:49, Thomas Mortagne <[email protected]> wrote: > > On Fri, Sep 2, 2016 at 10:00 AM, Vincent Massol <[email protected]> wrote: >> FWIW I’ve done a quick analysis of the speed of executing a XAR unit test >> (see http://dev.xwiki.org/xwiki/bin/view/Community/Testing#HXARTesting). >> >> Globally it takes 3ms for the first test on my machine. >> >> With a profiler the time is roughly doubled (6ms) and is spread like this >> (for stuff that takes the most time): >> >> * loadPage(): 1.1s >> ** TIKA init : 800ms (initialized in XWikiAttachment) >> >> * getRenderedContent(): 1.2s >> ** org.apache.xpath.jaxp.XPathImpl.evaluate(): 260ms >> ** javax.validation.Validation.buildDefaultValidatorFactory(): 260ms >> >> * init stuff: >> ** org.slf4j.LoggerFactory.getLogger(String): 804ms >> ** mocking: 288ms >> ** component registration: 260ms >> > > Note that except for component registration and mocking (not sure > about XPathImpl#evaluate), I think most of the others are about static > stuff being initialized the first time so following tests in the same > run should be way faster than the first one.
Yep, this is why I mentioned "Globally it takes 3s for the first test on my machine.”. The following tests take about 100+ms. But ATM we don’t have that many tests per test class for XAR testing. Right now we have 3 test classes: - 1 test class has 1 test - 1 test class has 2 tests - 1 test class has 4 tests So indeed in total it’s about 9.4s, which leads to an average of 1.3s per test. I still think it’s too important and takes too long to consider them unit tests and they should be in the category of integration tests. WDYT? Thanks -Vincent > My only ideas for speeding slightly the execution time would be to ensure > that TIKA is not initialized for the test until it is required. And maybe > check why SLF4J takes so long and find a way to make it run faster (maybe use > a noop logger). >> >> So those 2 items could win us a max of 1.6s out of 6s, so that’s 800ms out >> of 3s. Which would lower our exec time for a single test to close to 2 >> seconds. >> >> But I don’t see how to go below that value. >> >> Conclusion: >> ========== >> >> - IMO we shouldn’t call those tests unit tests but integration tests (that’s >> what they are in practice) >> - we should run them in the integration-tests profile to not slow down the >> main build too much >> - they’re still way faster than writing a functional tests (more than 1 >> minute to package and run - that’s 20 times slower at best), so when >> possible this should be favored when we want to validate some specific >> behavior of a wiki page or ensure that a regression won’t happen again >> >> WDYT? >> >> If we’re ok I’ll make the changes (integration-tests profile + rename to >> integration tests). >> >> Thanks >> -Vincent >> >> >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs > > > > -- > Thomas Mortagne > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
