> On 2 Jul 2018, at 10:58, Adel Atallah <adel.atal...@xwiki.com> wrote:
>
> Also I hope it will not make CI builds a lot longer.
It should make it slower to start the container but only marginally. What can
be slower is that it starts a VNC server which we were not starting before.
Globally that should add a couple of seconds.
Thanks
-Vincent
> Adel Atallah
> Product developer intern
> adel.atal...@xwiki.com
> tel: +33 (0)6 12 96 35 06
>
>
> On Mon, Jul 2, 2018 at 10:52 AM, Adel Atallah <adel.atal...@xwiki.com> wrote:
>> Hi,
>>
>> It will be very useful to be able to run functional tests on all
>> supported browsers. However I wonder if it will also be as useful for
>> different servlet containers and databases (at least when the database
>> schema doesn't change).
>> I tried to run the tests on my machine but I got an error:
>> https://pastebin.com/raw/cYjec5Bm (I haven't analyzed it yet but you
>> might have an idea why I'm getting this).
>> Also it's a bit disturbing to wait for the docker images to be
>> downloaded without knowing it (we should at least have a message
>> saying "Downloading images..." or even better, the docker output).
>>
>> Thanks,
>> Adel
>>
>>
>> On Mon, Jul 2, 2018 at 10:40 AM, Vincent Massol <vinc...@massol.net> wrote:
>>> Important note:
>>> * We’ll only be able to test Firefox and Chrome browsers since there’s no
>>> Docker containers for windows/mac having IE11/Edge and Safari (they’re not
>>> supported ATM even on windows machines).
>>> * We could work around this by having some agents running on Windows and
>>> using the installed IE11/Edge browser (i.e. not running in a Docker
>>> container). However it’s too complex FTM and I’d instead move this to later
>>> and instead focus on running XWiki on various DBs.
>>> * So this means that we’ll still need manual tests for IE11/Edge/Safari for
>>> the moment...
>>>
>>> Thanks
>>> -Vincent
>>>
>>>> On 30 Jun 2018, at 14:59, Vincent Massol <vinc...@massol.net> wrote:
>>>>
>>>> Status as of today:
>>>>
>>>> * I have a working POC for running XWiki functional tests in various
>>>> browsers (chrome and firefox FTM), using the latest Selenium and latest
>>>> Chrome/Firefox. I’ll commit today or in the coming days. Ofc it requires
>>>> docker to be installed so I’ll create a “docker” profile in the build so
>>>> that the build doesn’t break on agents which don’t have docker installed
>>>> yet. It’ll allow you to start testing it on your local machines.
>>>>
>>>> * I have updated the design page at
>>>> http://design.xwiki.org/xwiki/bin/view/Proposal/ConfigurationTesting
>>>>
>>>> Next steps:
>>>>
>>>> * Commit the work one (in the coming days)
>>>> * I have started trying to deploy XWiki in an embedded undertow. Still not
>>>> working at this stage (but I’ve progressed). I still don’t know if this is
>>>> going to work or not. I hope it will since that would save me from having
>>>> to write the equivalent of the Packager plugin.
>>>> * Integrate ability to choose the DB to use for the tests.
>>>>
>>>> Feedback most welcome.
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>> On 27 May 2018, at 21:44, Vincent Massol <vinc...@massol.net> wrote:
>>>>>
>>>>> Hi devs,
>>>>>
>>>>> I’d like to give you some info about what I’ve started working on and
>>>>> verify you like the direction I’m proposing to take for the future of
>>>>> functional testing on the xwiki project.
>>>>>
>>>>> Needs
>>>>> =====
>>>>>
>>>>> * Be able to test xwiki on multiple environments
>>>>>
>>>>> Context
>>>>> ======
>>>>>
>>>>> * Right now we test only in 1 env (Jetty+HSQLDB)
>>>>> * I've started some docker images in xwiki-contrib
>>>>> * I’ve also started some experiment through
>>>>> https://jira.xwiki.org/browse/XWIKI-14929 and
>>>>> https://jira.xwiki.org/browse/XWIKI-14930 (see also email thread
>>>>> "[Brainstorming] Implementing multi-environment tests - Take 2” and
>>>>> https://github.com/xwiki/xwiki-platform/compare/XWIKI-14929-14930). This
>>>>> email supersedes the "[Brainstorming] Implementing multi-environment
>>>>> tests - Take 2” thread.
>>>>> * Initially I imagined doing the multi env testing in Jenkins thanks to
>>>>> the Jenkins Docker plugin/library. However I realized that it would be
>>>>> better to be able to run that on the dev machines and thus decided
>>>>> instead to implement it at the maven level thanks to the Fabric8 Maven
>>>>> plugin.
>>>>>
>>>>> Proposal
>>>>> =======
>>>>>
>>>>> * The new proposal is to stop trying to do it at the maven level and
>>>>> instead do it at the Java level, i.e. be able to control (start/stop the
>>>>> various docker images for the DB, Servlet Container/XWiki and the Browser
>>>>> from within the java junit/selenium tests).
>>>>> * There are several java libraries existing to control docker from within
>>>>> java. For example: https://github.com/docker-java/docker-java
>>>>> * I got convinced when finding this awesome library that combines
>>>>> JUnit5/Selenium and Docker for multi-browser testing:
>>>>> https://bonigarcia.github.io/selenium-jupiter/
>>>>> ** Note that this relies on the browser docker images provided by the
>>>>> Selenoid project: https://aerokube.com/selenoid/latest/
>>>>> * So the idea is to extend that to be able to control the other 2 docker
>>>>> containers for the DB + ServletContainer/XWiki.
>>>>>
>>>>> Pros
>>>>> ====
>>>>>
>>>>> * Very simple setup to start/stop functional tests (and to debug them).
>>>>> Only requires Docker to be installed locally.
>>>>> * Very simple to test any combination of DB/Servlet Container/Browser.
>>>>> * Always up to date images with the latest version (we can depend on
>>>>> LATEST of Browser images, MySQL, Tomcat, etc).
>>>>> * Using JUnit5 and thus the latest features
>>>>> * Moving to the latest Selenium version too
>>>>> * Also supports manually executing tests in a given running xwiki instance
>>>>>
>>>>> Implementation
>>>>> ============
>>>>>
>>>>> Something like:
>>>>>
>>>>> --> XWikiSeleniumExtension extends SeleniumExtension
>>>>>
>>>>> @ExtendWith(XWikiSeleniumExtension.class)
>>>>> public class Test
>>>>>
>>>>> @Test
>>>>> public void xxx(XWikiWebDriver driver)
>>>>> {
>>>>> …
>>>>> }
>>>>>
>>>>> And be able to configure the DB to use, the Servlet container to use, and
>>>>> the packaging to use from system properties (and also from the test
>>>>> itself, see
>>>>> https://bonigarcia.github.io/selenium-jupiter/#generic-driver).
>>>>>
>>>>> The idea is to reimplement the XWiki Packaging Maven plugin as a java lib
>>>>> using Aether and to just start our functional tests using pure junit
>>>>> without anymore more. All the hard work will be performed by the JUnit5
>>>>> extension (create the packaging if not already exist, update some part of
>>>>> it if files have been modifier, start/stop DB+Servlet+Browser+Selenium,
>>>>> download the docker images).
>>>>>
>>>>> The packaging will be configurable. Some ideas of options:
>>>>> * use an already running xwiki instance
>>>>> * docker created from full XS zip from URL
>>>>> * docker created from XS zip from maven artifact
>>>>> * docker created from computed based on pom in current dir
>>>>>
>>>>> Migration
>>>>> =======
>>>>>
>>>>> Once a first version is working, it’ll be easy to use it only for a
>>>>> single platform functional tests and then slowly move each module to use
>>>>> the new way for its functional tests.
>>>>>
>>>>> WDYT?
>>>>>
>>>>> I’m planning to continue my investigation/development of this. So please
>>>>> let me know if you have feedback.
>>>>>
>>>>> Thanks
>>>>> -Vincent
>>>>>
>>>>>
>>>>
>>>
