Status as of today: * I have a working POC for running XWiki functional tests in various browsers (chrome and firefox FTM), using the latest Selenium and latest Chrome/Firefox. I’ll commit today or in the coming days. Ofc it requires docker to be installed so I’ll create a “docker” profile in the build so that the build doesn’t break on agents which don’t have docker installed yet. It’ll allow you to start testing it on your local machines.
* I have updated the design page at http://design.xwiki.org/xwiki/bin/view/Proposal/ConfigurationTesting Next steps: * Commit the work one (in the coming days) * I have started trying to deploy XWiki in an embedded undertow. Still not working at this stage (but I’ve progressed). I still don’t know if this is going to work or not. I hope it will since that would save me from having to write the equivalent of the Packager plugin. * Integrate ability to choose the DB to use for the tests. Feedback most welcome. Thanks -Vincent > On 27 May 2018, at 21:44, Vincent Massol <[email protected]> wrote: > > Hi devs, > > I’d like to give you some info about what I’ve started working on and verify > you like the direction I’m proposing to take for the future of functional > testing on the xwiki project. > > Needs > ===== > > * Be able to test xwiki on multiple environments > > Context > ====== > > * Right now we test only in 1 env (Jetty+HSQLDB) > * I've started some docker images in xwiki-contrib > * I’ve also started some experiment through > https://jira.xwiki.org/browse/XWIKI-14929 and > https://jira.xwiki.org/browse/XWIKI-14930 (see also email thread > "[Brainstorming] Implementing multi-environment tests - Take 2” and > https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930). This > email supersedes the "[Brainstorming] Implementing multi-environment tests - > Take 2” thread. > * Initially I imagined doing the multi env testing in Jenkins thanks to the > Jenkins Docker plugin/library. However I realized that it would be better to > be able to run that on the dev machines and thus decided instead to implement > it at the maven level thanks to the Fabric8 Maven plugin. > > Proposal > ======= > > * The new proposal is to stop trying to do it at the maven level and instead > do it at the Java level, i.e. be able to control (start/stop the various > docker images for the DB, Servlet Container/XWiki and the Browser from within > the java junit/selenium tests). > * There are several java libraries existing to control docker from within > java. For example: https://github.com/docker-java/docker-java > * I got convinced when finding this awesome library that combines > JUnit5/Selenium and Docker for multi-browser testing: > https://bonigarcia.github.io/selenium-jupiter/ > ** Note that this relies on the browser docker images provided by the > Selenoid project: https://aerokube.com/selenoid/latest/ > * So the idea is to extend that to be able to control the other 2 docker > containers for the DB + ServletContainer/XWiki. > > Pros > ==== > > * Very simple setup to start/stop functional tests (and to debug them). Only > requires Docker to be installed locally. > * Very simple to test any combination of DB/Servlet Container/Browser. > * Always up to date images with the latest version (we can depend on LATEST > of Browser images, MySQL, Tomcat, etc). > * Using JUnit5 and thus the latest features > * Moving to the latest Selenium version too > * Also supports manually executing tests in a given running xwiki instance > > Implementation > ============ > > Something like: > > --> XWikiSeleniumExtension extends SeleniumExtension > > @ExtendWith(XWikiSeleniumExtension.class) > public class Test > > @Test > public void xxx(XWikiWebDriver driver) > { > … > } > > And be able to configure the DB to use, the Servlet container to use, and the > packaging to use from system properties (and also from the test itself, see > https://bonigarcia.github.io/selenium-jupiter/#generic-driver). > > The idea is to reimplement the XWiki Packaging Maven plugin as a java lib > using Aether and to just start our functional tests using pure junit without > anymore more. All the hard work will be performed by the JUnit5 extension > (create the packaging if not already exist, update some part of it if files > have been modifier, start/stop DB+Servlet+Browser+Selenium, download the > docker images). > > The packaging will be configurable. Some ideas of options: > * use an already running xwiki instance > * docker created from full XS zip from URL > * docker created from XS zip from maven artifact > * docker created from computed based on pom in current dir > > Migration > ======= > > Once a first version is working, it’ll be easy to use it only for a single > platform functional tests and then slowly move each module to use the new way > for its functional tests. > > WDYT? > > I’m planning to continue my investigation/development of this. So please let > me know if you have feedback. > > Thanks > -Vincent > >
