> On 16 Nov 2018, at 22:03, Vincent Massol <[email protected]> wrote:
>
> Hi devs,
>
> So we now have 2 dockers jobs:
> - one that executes daily and runs the latest version of the supported
> configs. It runs all docker tests (right now that’s MenuIT but I’ll start
> converting some other tests and we need to convert them all). See
> https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker/
> - another one that executes weekly and that runs all supported configs but
> only on smoke tests (right now that MenuIT only). See
> https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_all/
And to see which configs are tested and passing or not:
https://ci.xwiki.org/view/Recommended%20Builds/job/xwiki-platform_docker_all/lastCompletedBuild/testReport/org.xwiki.menu.test.ui/MenuIT/
Thanks
-Vincent
> All are passing FTM :)
>
> Thanks
> -Vincent
>
>> On 5 Nov 2018, at 17:22, Vincent Massol <[email protected]> wrote:
>>
>> Hi devs,
>>
>> I’m still not sure but FTM I was thinking of having 2 pipeline jobs:
>>
>> 1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on the
>> maximum number of configurations, in order to flesh out configs that don’t
>> start properly. For example XWiki on Tomcat 9.x would fail (since the Tomcat
>> 9.x docker image uses java9+). The job would not send a mail on failure but
>> it would update a report page (could even update a page on xwiki.org
>> directly or if too complex update some page on maven.xwiki.org somewhere).
>> This job would run not very often but say once per week. Note that one
>> config takes 3-4 minutes to run, so 50 configs would take 3 hours which is
>> acceptable.
>>
>> 2) Job 2: Execute all functional tests on a subset of supported configs. For
>> example we don’t need to run all the tests on PostgreSQL/Jetty/Chrome if we
>> already run on PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job will
>> take a long time to execute. We’ll start with 3-4 configs and will go to
>> about 10 configs when we add more. The tests will take roughly 2 hours to
>> execute per config I think. So a total of 20 hours when we have 10 configs.
>> If we run those once per week it should be fine.
>>
>> Note: Once we have job 1 & 2, we won't need to have the smoke tests I add as
>> part of the platform JenkinsFile.
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
>>
>
