Important note: * We’ll only be able to test Firefox and Chrome browsers since there’s no Docker containers for windows/mac having IE11/Edge and Safari (they’re not supported ATM even on windows machines). * We could work around this by having some agents running on Windows and using the installed IE11/Edge browser (i.e. not running in a Docker container). However it’s too complex FTM and I’d instead move this to later and instead focus on running XWiki on various DBs. * So this means that we’ll still need manual tests for IE11/Edge/Safari for the moment...
Thanks -Vincent > On 30 Jun 2018, at 14:59, Vincent Massol <[email protected]> wrote: > > Status as of today: > > * I have a working POC for running XWiki functional tests in various browsers > (chrome and firefox FTM), using the latest Selenium and latest > Chrome/Firefox. I’ll commit today or in the coming days. Ofc it requires > docker to be installed so I’ll create a “docker” profile in the build so that > the build doesn’t break on agents which don’t have docker installed yet. > It’ll allow you to start testing it on your local machines. > > * I have updated the design page at > http://design.xwiki.org/xwiki/bin/view/Proposal/ConfigurationTesting > > Next steps: > > * Commit the work one (in the coming days) > * I have started trying to deploy XWiki in an embedded undertow. Still not > working at this stage (but I’ve progressed). I still don’t know if this is > going to work or not. I hope it will since that would save me from having to > write the equivalent of the Packager plugin. > * Integrate ability to choose the DB to use for the tests. > > Feedback most welcome. > > Thanks > -Vincent > >> On 27 May 2018, at 21:44, Vincent Massol <[email protected]> wrote: >> >> Hi devs, >> >> I’d like to give you some info about what I’ve started working on and verify >> you like the direction I’m proposing to take for the future of functional >> testing on the xwiki project. >> >> Needs >> ===== >> >> * Be able to test xwiki on multiple environments >> >> Context >> ====== >> >> * Right now we test only in 1 env (Jetty+HSQLDB) >> * I've started some docker images in xwiki-contrib >> * I’ve also started some experiment through >> https://jira.xwiki.org/browse/XWIKI-14929 and >> https://jira.xwiki.org/browse/XWIKI-14930 (see also email thread >> "[Brainstorming] Implementing multi-environment tests - Take 2” and >> https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930). This >> email supersedes the "[Brainstorming] Implementing multi-environment tests - >> Take 2” thread. >> * Initially I imagined doing the multi env testing in Jenkins thanks to the >> Jenkins Docker plugin/library. However I realized that it would be better to >> be able to run that on the dev machines and thus decided instead to >> implement it at the maven level thanks to the Fabric8 Maven plugin. >> >> Proposal >> ======= >> >> * The new proposal is to stop trying to do it at the maven level and instead >> do it at the Java level, i.e. be able to control (start/stop the various >> docker images for the DB, Servlet Container/XWiki and the Browser from >> within the java junit/selenium tests). >> * There are several java libraries existing to control docker from within >> java. For example: https://github.com/docker-java/docker-java >> * I got convinced when finding this awesome library that combines >> JUnit5/Selenium and Docker for multi-browser testing: >> https://bonigarcia.github.io/selenium-jupiter/ >> ** Note that this relies on the browser docker images provided by the >> Selenoid project: https://aerokube.com/selenoid/latest/ >> * So the idea is to extend that to be able to control the other 2 docker >> containers for the DB + ServletContainer/XWiki. >> >> Pros >> ==== >> >> * Very simple setup to start/stop functional tests (and to debug them). Only >> requires Docker to be installed locally. >> * Very simple to test any combination of DB/Servlet Container/Browser. >> * Always up to date images with the latest version (we can depend on LATEST >> of Browser images, MySQL, Tomcat, etc). >> * Using JUnit5 and thus the latest features >> * Moving to the latest Selenium version too >> * Also supports manually executing tests in a given running xwiki instance >> >> Implementation >> ============ >> >> Something like: >> >> --> XWikiSeleniumExtension extends SeleniumExtension >> >> @ExtendWith(XWikiSeleniumExtension.class) >> public class Test >> >> @Test >> public void xxx(XWikiWebDriver driver) >> { >> … >> } >> >> And be able to configure the DB to use, the Servlet container to use, and >> the packaging to use from system properties (and also from the test itself, >> see https://bonigarcia.github.io/selenium-jupiter/#generic-driver). >> >> The idea is to reimplement the XWiki Packaging Maven plugin as a java lib >> using Aether and to just start our functional tests using pure junit without >> anymore more. All the hard work will be performed by the JUnit5 extension >> (create the packaging if not already exist, update some part of it if files >> have been modifier, start/stop DB+Servlet+Browser+Selenium, download the >> docker images). >> >> The packaging will be configurable. Some ideas of options: >> * use an already running xwiki instance >> * docker created from full XS zip from URL >> * docker created from XS zip from maven artifact >> * docker created from computed based on pom in current dir >> >> Migration >> ======= >> >> Once a first version is working, it’ll be easy to use it only for a single >> platform functional tests and then slowly move each module to use the new >> way for its functional tests. >> >> WDYT? >> >> I’m planning to continue my investigation/development of this. So please let >> me know if you have feedback. >> >> Thanks >> -Vincent >> >> >
