[
https://jira.nuxeo.com/browse/NXBT-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Carsique updated NXBT-348:
---------------------------------
Component/s: Selenium tests
Continuous integration
Tags: Backlog
> Improve CI and Testing framework
> --------------------------------
>
> Key: NXBT-348
> URL: https://jira.nuxeo.com/browse/NXBT-348
> Project: Nuxeo ECM Build/Test Environment
> Issue Type: User story
> Components: Continuous integration, Selenium tests
> Reporter: Thierry Delprat
> Assignee: Julien Carsique
>
> h2. Technical debt on Test
> h3. Non Regression tests
> - rewrite Selenium tests in WebDriver
> => make them more modular and easier to debug
> - improve Nuxeo HTML layout to make tests easier
> => make tests more robust
> h3. Funkload tests
> Currently only Ben can really fix a broken test.
> We should probably improve training and documentation so that not only Ben
> can fix them when needed.
> => migrate CMIS and Rest Tests in java
> h3. Multi-DB / Multi-OS
> MultiDB and Multi-OS tests matrix have been added recently.
> There is still a lot of work to be done so that they all run :
> - fix unit tests on MSSQL and MySQL (h2, PGSQL et Oracle are ok)
> - Windows : there are still failed unit tests under windows
> - OpenJDK 7 : there are still failed unit tests under windows
> Non regression testing in Matrix should also be fixed but it's probably
> pointless until all unit tests are fixed.
> h2. CI infrastructure speedup
> The goal is to make CI feedback quicker.
> We can not expect CI feedback to take a only few minutes :
> - build is big
> - we have a lot of tests in different configuration
> But there are some actions we can do to improve the feedback time.
> h3. fix Nexus / Jenkins concurrency issue
> Nexus and Jenkins master share the same server and the same NAS.
> - Jenkins Master uses NAS storage to store the workspace (build and test
> results)
> - Nexus uses NAS to store the repo
> Monitoring shows that the server has a load average about 2 and that Jenkins
> or Nexus are faster when the other service is off.
> This means we should work on the global optimization of this Nexus/Jenkins
> setup : that will globalling improve speed of all CI :
> - for fetching / pushing artifacts
> - for storing results
> Based on Ben's remarks, the main lead for improvement seems to be the NFS
> share :
> - separate NASs for Jenkins / Nexus ?
> - use local storage instead of NAS ?
> - NFS / NAS tunning ?
> - ...
> h3. GWT Build issue
> GWT build is really very slow and represent a big part of the global Nuxeo
> build time event if we only have a few (3 or 4) GWT modules.
> => This needs to be fixed, both for CI and for developpers
> Identified solutions include :
> - change profile to limit the number of target browser (meaning nb of
> permitation) during the standard Build
> ( we will still need to do the full build for releases)
> - disable GWT optimizations during standard builds
> - be sure to only trigger the GWT build when needed
> See
> http://stackoverflow.com/questions/1011863/how-do-i-speed-up-the-gwt-compiler
> NB : we can also extract GWT modules in Addons to help running GWT builds in
> parallel
> h3. Slave build performances
> Currently the build on the Jenkins slaves is about the same time than on a
> dev laptop.
> => we can make better !
> It's true that a big part of the Jenkins slave are poor hardware (the dwarf).
> Before taking any decision, we should probably do some profiling to see
> exactly what are the real impacting factors :
> - test build on a bigger CPU (like Octo)
> - use RAMFS to speedup I/O
> - tweak compile options ?
> - test on big RackSpace VMs
> => based on that we will be able to define what kind of Build slaves we need
> (then we will have to see what we can host internaly and what must be
> external)
> h2. CI Config
> h3. Permanent checks
> Permanent checks currently don't wait for nuxeo-features to be built but
> won't raise relevant issues until nuxeo-features is built.
> In cases where there are a lot of commits, nuxeo-features build can take
> several hours.
> => need to change that
> So basically, we could have permanent checks be closer to the IT builds :
> - rebuild everything from zero
> - do build / no UT / distrib on a fast slave
> - dispatch non-reg tests on different hosts
> h3. Addons
> Current distribution build depends on some addons: the one that are
> considered standard and used in distribs / MP packages.
> => Create a job that build all "standard" addons so that distribution does
> not have to wait for 20 jobs
> h3. JBoss distrib
> We need to update JBoss distribution to align on tomcat/MP build.
> h2. Tests to add
> There are still some part of the platform that are poorly / not enough tested
> h3. Studio
> XXX Let Anahide list what she has in mind, but basically :
> - define a standard Studio project
> - run build against different targets
> - check result in a running Nuxeo on several target versions
> h3.Admin Center / Marketplace
> We already have a webdriver test that tests the basis.
> We need to extend it so that we validate for example :
> - add CMF / uninstall DM
> - Install DAM / uninstall DAM + install CMF
> - ...
> XXX more here
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
_______________________________________________
ECM-tickets mailing list
ECM-tickets@lists.nuxeo.com
http://lists.nuxeo.com/mailman/listinfo/ecm-tickets
