Hello Morgan, Would it be possible to switch to OPNFV Functest latest containers (several runs) ? We have updated all OPNFV Functest test cases for Gambia Release (N+1) and we need a reference (and stable) platform to get a real overview.
From the time being, only Compass is currently providing interesting results (most installers are only running OPNFV Functest Fraser). It seems that we are getting the same results between OPNFV Functest Fraser and Gambia (e.g. highlighting same bugs about Compass). https://build.opnfv.org/ci/job/functest-compass-baremetal-daily-master/92/console (os-nosdn-nofeature-ha) https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-master/88/console (os-odl_l3-nofeature-ha) It should be noted that refstack_defcore is temporarily disabled. It simply doesn’t meet OpenStack Queens requirements (refstack-client is hardcoded to testr). We could expect a full success vs Orange ONAP Openlab which is already fully verified by OPNFV Functest Fraser. https://wiki.opnfv.org/pages/viewpage.action?pageId=13211751 Even if OpenStack Pike has been deployed, it should be fine thanks to backward compatibility enforced by OpenStack. Thank you in advance for your help, Cédric De : OLLIVIER Cédric IMT/OLN Envoyé : jeudi 12 avril 2018 14:16 À : RICHOMME Morgan IMT/OLN; dmcbr...@linuxfoundation.org; opnfv-tech-discuss@lists.opnfv.org Cc : helen.c...@huawei.com; DEBEAU Eric IMT/OLN; gabriel.yuy...@huawei.com Objet : RE: [OPNFV] [Fraser] Fraser Functest run on Orange ONAP Open lab Hello Morgan, Thank you very much for your help. All tests done via ONAP OpenLab has been very helpful throughout the F-release by providing the quickest feedbacks. We are checking if Functest testcases have to complete Snaps’ garbage collectors for floating ips. That would explain this remaining resources. Yes, we will target live testing for the next release. http://testresults.opnfv.org/functest/gambiachallenges/ About the reuse of OPNFV in ONAP, Xtesting has been synchronized to Queens this morning (opnfv/xtesting:latest). Onap Healthcheck container could inherit from opnfv/xtesting:fraser if it needs requirements synchronized with OpenStack Pike. Functest (master) could switch to Queens as well in order to get first feedbacks before next MS0. https://gerrit.opnfv.org/gerrit/#/c/55553/ Cédric De : RICHOMME Morgan IMT/OLN Envoyé : mardi 10 avril 2018 09:52 À : dmcbr...@linuxfoundation.org<mailto:dmcbr...@linuxfoundation.org>; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>; OLLIVIER Cédric IMT/OLN Cc : helen.c...@huawei.com<mailto:helen.c...@huawei.com>; DEBEAU Eric IMT/OLN; gabriel.yuy...@huawei.com<mailto:gabriel.yuy...@huawei.com> Objet : [OPNFV] [Fraser] Fraser Functest run on Orange ONAP Open lab Hi for information, we switched to the fraser version for our daily test on our ONAP OpenLab, a communit lab for ONAP users. Even if the lab is used by ONAP contributors for ONAP testing, we are reusing Functest test suites to be sure that the infrastructure supporting ONAP is still properly working. We were using latest tag so far and decided to go to fraser to provide some feedback for the OPNFV forthcoming release. The infrastructure has been deployed using XCI/bifrost scripts + Kolla installer (currently under integration in XCI) on bare metal. The scenario corresponds to a basic os-nosdn-nofeature-ha scenario. the results are perfect :) +----------------------------+------------------+---------------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------------+------------------+---------------------+------------------+----------------+ | connection_check | functest | healthcheck | 00:07 | PASS | | api_check | functest | healthcheck | 08:26 | PASS | | snaps_health_check | functest | healthcheck | 00:49 | PASS | +----------------------------+------------------+---------------------+------------------+----------------+ +------------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +------------------------------+------------------+---------------+------------------+----------------+ | vping_ssh | functest | smoke | 00:42 | PASS | | vping_userdata | functest | smoke | 00:35 | PASS | | tempest_smoke_serial | functest | smoke | 12:23 | PASS | | rally_sanity | functest | smoke | 20:46 | PASS | | refstack_defcore | functest | smoke | 06:53 | PASS | | patrole | functest | smoke | 04:29 | PASS | | snaps_smoke | functest | smoke | 50:18 | PASS | | odl | functest | smoke | 00:00 | SKIP | | odl_netvirt | functest | smoke | 00:00 | SKIP | | neutron_trunk | functest | smoke | 00:00 | SKIP | +------------------------------+------------------+---------------+------------------+----------------+ +----------------------+------------------+--------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +----------------------+------------------+--------------+------------------+----------------+ | cloudify_ims | functest | vnf | 27:40 | PASS | | vyos_vrouter | functest | vnf | 18:23 | PASS | | juju_epc | functest | vnf | 51:17 | PASS | +----------------------+------------------+--------------+------------------+----------------+ Thanks to the functest and XCI team! Tests will be run daily so we would report any issue if we are facing some but it looks pretty stable. Unlike the traditionnal approach, we are running daily tests on a up&running infrastructure (we do not reinstall it systematically before launching the tests) so it is connected somehow to the long duration test acctivities coordinated by Gabriel The only issues we faced so far with latest version were due to the cleaning of the resources (shortages of floatting IPs) This issue has been identified and will be addressed as far as I understood in Gambia release. we run now a cleaning script after the tests to free some resources As a conclusion, the Orange Onap OpenLab validates the Fraser scenario os-nosdn-nofeature-ha for XCI/Kolla with functest healthcheck/smoke/vnf (duration 3h and 20 minutes) It could be interesting for us to add Yardstick daily tests...it is in the todo list... Morgan _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
