Martin, Thanks for the feedback and insight. It looks like lots of good pointers and observations. We have a mixed level of CI experience, so your input will surely be valuable.
We have a limited amount of dedicated hardware for pods in this lab (at UNH). Currently we are limited to one bare metal arm64-based pod (6 servers - jump and 5 nodes). There are other resources available (a good number of beefy servers of both x86 and arm64 architecture) in the UNH lab, but it's all LaaS, i.e. short-term subscriptions, and they're currently limited to one server at a time. However, the LaaS engineers are adding multi-server pod capability to the automated LaaS dashboard...may be available in time for us to use as additional bare metal pods. These will be (like the single servers) short term subscriptions (i.e. 3 week term with up to 2 1-week extensions, then you have to return to the free pool). Welcome to Auto! Joe On Mon, Jun 4, 2018 at 10:39 AM, Klozik Martin <martin.klo...@tieto.com> wrote: > Hi Auto Team, > > > I've read the CI related wiki page at https://wiki.opnfv.org/ > display/AUTO/CI+for+Auto and I would like to discuss with you a few > thoughts based on my experience with CI from OPNFV vswitchperf project. > > > > - I agree, that a dedicated POD should be reserved for Auto CI. Based > on the length of the job execution and the content of VERIFY & MERGE jobs, > it might be required to introduce an additional (2nd) POD in the future. I > would prefer a bare metal POD to virtual one to simplify the final setup > and thus improve its robustness. > > > - Based on the configuration, daily jobs can be triggered daily or > "daily if there was any commit since the last CI run" (pollSCM trigger). > The second setup can easy the load at CI POD - useful in case that POD is > shared among several projects or jobs, e.g. more compute power is left for > VERIFY/MERGE jobs. I suppose that in case of Auto the frequency will be > really daily to constantly verify OPNFV & ONAP (master branch) > functionality even if Auto repository was not modified. In case of stable > release it doesn't make sense to run it on daily basis as OPNFV & ONAP > versions will be fixed. > - I agree with the split of functionality between job definition yaml > file (in releng repo) and real job "body" inside a Auto repo. This will add > a flexibility to the job configuration and it will also allow us to verify > new changes as part of verify job during review process before the changes > are merged. For example, VSPERF project uses a CI related script > ci/build-vsperf.sh, which based on the parameter executes daily, verify or > merge job actions. This script is then called from jenkins yaml file ( > jjb/vswitchperf/vswitchperf.yaml) with appropriate parameter. > - In case of auto CI, it might be useful to split CI job into the set > of dependent jobs, where following job will be executed only in case that > previous job has succeeded. This will simplify analysis of job failures as > it will be visible for the first sight if the failure is caused by Auto TCs > or by platform installers. The history of these jobs will give us a simple > overview of "sub-jobs" stability. > > E.g. > auto-daily-master > |---> auto-install-opnfv > |----> auto-install-onap > |-----> auto-daily-tests > > example 2: > auto-verify-master > |---> auto-install-opnfv > |----> auto-install-onap > |-----> auto-verify > code validation... OK > doc validation .... OK > sanity TC .... OK > > > - It might be useful to prepare some output filtering. Otherwise it > would be difficult to read the jenkins job console log during analysis of > CI run (especially a failure). It means to suppress console output of > successful steps or tests and print only simplified results (as shown in > auto-verify example above). This approach expects, that full logs are > either accessible (i.e. kept for a while) at POD or dumped into jenkins job > console output in case that error is detected. > - there are three basic job types used in OPNFV - DAILY, VERIFY > (triggered by gerrit for every pushed patch or its change) and MERGE > (triggered by gerrit after the merge of the patch). I suppose that in case > of AUTO, VERIFY & MERGE jobs can share the content. > - Do you plan to store any artifacts "generated" by CI job into OPNFV > artifactory? For example, vswitchperf is pushing results into OPNFV results > database (I've seen, that AUTO is also going to push to results DB ) > and after that it stores appropriate log files and test report into > artifacts.opnfv.org. Does it make sense to store logs from OPNFV/ONAP > installation or execution of Auto TCs into artifactory too? > - It might be useful to easily switch/test different OPNFV/ONAP > versions. For example a "master" CI script can import a versions.sh file > where user/tester can specify OPNFV and ONAP branches or commit IDs to be > used. > - Analogically to previous point, also the OPNFV installer and ONAP > deployment methods can be specified in the (same?) file. > > > Please consider these thoughts as an "outsider" point of view at Auto CI, > as I'm new to the Auto project. It is highly possible that some of these > points were already discussed or even solved already. In that case please > don't hesitate to point out the sources I should check. Based on the > discussion, I'll take care about CI auto wiki page updates, if it will be > needed. > > Thank you, > Martin > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > >
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
