+1 for both but we need to push for more powerful CI agents. ATM the build is faster on my laptop.
Thanks, Marius On Mon, Nov 5, 2018 at 6:22 PM Vincent Massol <[email protected]> wrote: > Hi devs, > > I’m still not sure but FTM I was thinking of having 2 pipeline jobs: > > 1) Job 1: Execute one functional test only (e.g. MenuIT for now) but on > the maximum number of configurations, in order to flesh out configs that > don’t start properly. For example XWiki on Tomcat 9.x would fail (since the > Tomcat 9.x docker image uses java9+). The job would not send a mail on > failure but it would update a report page (could even update a page on > xwiki.org directly or if too complex update some page on maven.xwiki.org > somewhere). This job would run not very often but say once per week. Note > that one config takes 3-4 minutes to run, so 50 configs would take 3 hours > which is acceptable. > > 2) Job 2: Execute all functional tests on a subset of supported configs. > For example we don’t need to run all the tests on PostgreSQL/Jetty/Chrome > if we already run on PostgreSQL/Tomcat/FF and MySQL/Tomcat/Chrome. This job > will take a long time to execute. We’ll start with 3-4 configs and will go > to about 10 configs when we add more. The tests will take roughly 2 hours > to execute per config I think. So a total of 20 hours when we have 10 > configs. If we run those once per week it should be fine. > > Note: Once we have job 1 & 2, we won't need to have the smoke tests I add > as part of the platform JenkinsFile. > > WDYT? > > Thanks > -Vincent > >
