Also I hope it will not make CI builds a lot longer. Adel Atallah Product developer intern [email protected] tel: +33 (0)6 12 96 35 06
On Mon, Jul 2, 2018 at 10:52 AM, Adel Atallah <[email protected]> wrote: > Hi, > > It will be very useful to be able to run functional tests on all > supported browsers. However I wonder if it will also be as useful for > different servlet containers and databases (at least when the database > schema doesn't change). > I tried to run the tests on my machine but I got an error: > https://pastebin.com/raw/cYjec5Bm (I haven't analyzed it yet but you > might have an idea why I'm getting this). > Also it's a bit disturbing to wait for the docker images to be > downloaded without knowing it (we should at least have a message > saying "Downloading images..." or even better, the docker output). > > Thanks, > Adel > > > On Mon, Jul 2, 2018 at 10:40 AM, Vincent Massol <[email protected]> wrote: >> Important note: >> * We’ll only be able to test Firefox and Chrome browsers since there’s no >> Docker containers for windows/mac having IE11/Edge and Safari (they’re not >> supported ATM even on windows machines). >> * We could work around this by having some agents running on Windows and >> using the installed IE11/Edge browser (i.e. not running in a Docker >> container). However it’s too complex FTM and I’d instead move this to later >> and instead focus on running XWiki on various DBs. >> * So this means that we’ll still need manual tests for IE11/Edge/Safari for >> the moment... >> >> Thanks >> -Vincent >> >>> On 30 Jun 2018, at 14:59, Vincent Massol <[email protected]> wrote: >>> >>> Status as of today: >>> >>> * I have a working POC for running XWiki functional tests in various >>> browsers (chrome and firefox FTM), using the latest Selenium and latest >>> Chrome/Firefox. I’ll commit today or in the coming days. Ofc it requires >>> docker to be installed so I’ll create a “docker” profile in the build so >>> that the build doesn’t break on agents which don’t have docker installed >>> yet. It’ll allow you to start testing it on your local machines. >>> >>> * I have updated the design page at >>> http://design.xwiki.org/xwiki/bin/view/Proposal/ConfigurationTesting >>> >>> Next steps: >>> >>> * Commit the work one (in the coming days) >>> * I have started trying to deploy XWiki in an embedded undertow. Still not >>> working at this stage (but I’ve progressed). I still don’t know if this is >>> going to work or not. I hope it will since that would save me from having >>> to write the equivalent of the Packager plugin. >>> * Integrate ability to choose the DB to use for the tests. >>> >>> Feedback most welcome. >>> >>> Thanks >>> -Vincent >>> >>>> On 27 May 2018, at 21:44, Vincent Massol <[email protected]> wrote: >>>> >>>> Hi devs, >>>> >>>> I’d like to give you some info about what I’ve started working on and >>>> verify you like the direction I’m proposing to take for the future of >>>> functional testing on the xwiki project. >>>> >>>> Needs >>>> ===== >>>> >>>> * Be able to test xwiki on multiple environments >>>> >>>> Context >>>> ====== >>>> >>>> * Right now we test only in 1 env (Jetty+HSQLDB) >>>> * I've started some docker images in xwiki-contrib >>>> * I’ve also started some experiment through >>>> https://jira.xwiki.org/browse/XWIKI-14929 and >>>> https://jira.xwiki.org/browse/XWIKI-14930 (see also email thread >>>> "[Brainstorming] Implementing multi-environment tests - Take 2” and >>>> https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930). This >>>> email supersedes the "[Brainstorming] Implementing multi-environment tests >>>> - Take 2” thread. >>>> * Initially I imagined doing the multi env testing in Jenkins thanks to >>>> the Jenkins Docker plugin/library. However I realized that it would be >>>> better to be able to run that on the dev machines and thus decided instead >>>> to implement it at the maven level thanks to the Fabric8 Maven plugin. >>>> >>>> Proposal >>>> ======= >>>> >>>> * The new proposal is to stop trying to do it at the maven level and >>>> instead do it at the Java level, i.e. be able to control (start/stop the >>>> various docker images for the DB, Servlet Container/XWiki and the Browser >>>> from within the java junit/selenium tests). >>>> * There are several java libraries existing to control docker from within >>>> java. For example: https://github.com/docker-java/docker-java >>>> * I got convinced when finding this awesome library that combines >>>> JUnit5/Selenium and Docker for multi-browser testing: >>>> https://bonigarcia.github.io/selenium-jupiter/ >>>> ** Note that this relies on the browser docker images provided by the >>>> Selenoid project: https://aerokube.com/selenoid/latest/ >>>> * So the idea is to extend that to be able to control the other 2 docker >>>> containers for the DB + ServletContainer/XWiki. >>>> >>>> Pros >>>> ==== >>>> >>>> * Very simple setup to start/stop functional tests (and to debug them). >>>> Only requires Docker to be installed locally. >>>> * Very simple to test any combination of DB/Servlet Container/Browser. >>>> * Always up to date images with the latest version (we can depend on >>>> LATEST of Browser images, MySQL, Tomcat, etc). >>>> * Using JUnit5 and thus the latest features >>>> * Moving to the latest Selenium version too >>>> * Also supports manually executing tests in a given running xwiki instance >>>> >>>> Implementation >>>> ============ >>>> >>>> Something like: >>>> >>>> --> XWikiSeleniumExtension extends SeleniumExtension >>>> >>>> @ExtendWith(XWikiSeleniumExtension.class) >>>> public class Test >>>> >>>> @Test >>>> public void xxx(XWikiWebDriver driver) >>>> { >>>> … >>>> } >>>> >>>> And be able to configure the DB to use, the Servlet container to use, and >>>> the packaging to use from system properties (and also from the test >>>> itself, see https://bonigarcia.github.io/selenium-jupiter/#generic-driver). >>>> >>>> The idea is to reimplement the XWiki Packaging Maven plugin as a java lib >>>> using Aether and to just start our functional tests using pure junit >>>> without anymore more. All the hard work will be performed by the JUnit5 >>>> extension (create the packaging if not already exist, update some part of >>>> it if files have been modifier, start/stop DB+Servlet+Browser+Selenium, >>>> download the docker images). >>>> >>>> The packaging will be configurable. Some ideas of options: >>>> * use an already running xwiki instance >>>> * docker created from full XS zip from URL >>>> * docker created from XS zip from maven artifact >>>> * docker created from computed based on pom in current dir >>>> >>>> Migration >>>> ======= >>>> >>>> Once a first version is working, it’ll be easy to use it only for a single >>>> platform functional tests and then slowly move each module to use the new >>>> way for its functional tests. >>>> >>>> WDYT? >>>> >>>> I’m planning to continue my investigation/development of this. So please >>>> let me know if you have feedback. >>>> >>>> Thanks >>>> -Vincent >>>> >>>> >>> >>
