Hi I uploaded the slidedeck used for the demo I added some screenshots https://wiki.onap.org/pages/viewpage.action?pageId=6593670&preview=/6593670/25433913/integration_demo_E2E_chain.pdf
I got the action point to organize a follow-up on this topic through another meeting I am not sure I have the rights to book a zoom for an ad-hoc session (my personnal account allows only 40 minutes sessions) I tried to ask the question on the IRC chan but it is not very active and I did not find any guideline on the wiki (could make sense to add one page). @Gildas could you help me to book such slot? I could suggest tomorrow or Thursday 2h30 UTC (same time than the integration meeting) Some open questions need LF helpdesks view (see slide "next steps"). BTW I created a ticket to ONAP heldesk on the 5th of February but did not get any feedback so far (https://rt.linuxfoundation.org/SelfService/Display.html?id=52001) Did I miss something on the procedure regarding ONAP helpdesk? Meanwhile I wondered if it would be possible to mutualize things if OPNFV community agrees (+Tim, Gabriel, Ray, Heather and Aric in CC) The testresults.opnfv.org machine could be shared (everything is already there DB, dockers, mechanism to backup the DB) and a testresults.onap.org subdomain could be easily created (hopefully) and configured The VM would not suffer... [morgan@gce-opnfv-sandbox-fbrockners<mailto:morgan@gce-opnfv-sandbox-fbrockners> ~]$ df -h Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur /dev/sda1 75G 29G 47G 38% / [morgan@gce-opnfv-sandbox-fbrockners<mailto:morgan@gce-opnfv-sandbox-fbrockners> ~]$ free -m total used free shared buff/cache available Mem: 7480 3118 320 368 4040 3451 /Morgan Le mardi 13 février 2018 à 16:10 +0000, Tina Tsou a écrit : Dear Morgan, This is Tina Tsou, PTL of Auto Project. I was in Integration Project meeting just now. We are interested at your option #1 for the test API: integration OPNFV - ONAP. Would you send the meeting invitation to opnfv-tech-discussion too? So Auto participants can join and learn from you. Thank you, Tina On Feb 13, 2018, at 6:38 AM, Roger Maitland <roger.maitl...@amdocs.com<mailto:roger.maitl...@amdocs.com>> wrote: One clarification on OOM – it’s not a just a deployment tool, it manages the entire lifecycle of ONAP as shown in our logo. Cheers, Roger <image001.png> From: onap-discuss-boun...@lists.onap.org<mailto:onap-discuss-boun...@lists.onap.org> [mailto:onap-discuss-boun...@lists.onap.org] On Behalf Of morgan.richo...@orange.com<mailto:morgan.richo...@orange.com> Sent: Tuesday, February 13, 2018 2:41 AM To: bryan.sulli...@research.att.com<mailto:bryan.sulli...@research.att.com>; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Cc: feng.xiao...@zte.com.cn<mailto:feng.xiao...@zte.com.cn>; onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org> Subject: Re: [onap-discuss] [Auto] RE: [opnfv-tech-discuss] [integration] demo E2E testing & reporting next Tuesday? Hi Bryan, we can discuss it after the demo or offline As far as I understand Auto is the equivalent of OOM, i.e. a deployment of ONAP based on kubernetes (but Auto is on OPNFV side) - do not hesitate to correct me if I am wrong it would be possible to indicate Auto as 1 of the ONAP installer and collect the results of the tests that is one of the interests to use the test API/Test DB independently from jenkins. Several CI chains can push results to provide an aggregate view of E2E Functional and later performance testing on ONAP. and depending on the installer it is possible to customize the test list. For example if you have additional test cases through VES or Model projects. /Morgan Le vendredi 09 février 2018 à 16:54 +0000, SULLIVAN, BRYAN L (BRYAN L) a écrit : Auto team, FYI in case you have not seen this. Some of you may be collaborating with Morgan on the ONAP lab deployment, if not then perhaps there are some deployment tools you can leverage, or other ways that Orange can help the Auto project get off the ground with ONAP deployment for Auto use case development/testing. For those cc’d who may not yet be aware of the Auto project: https://wiki.opnfv.org/pages/viewpage.action?pageId=12389095 Other OPNFV project focused on functional capability / use case testing which will support and leverage the OPNFV-ONAP integration through the Auto project include VES (https://wiki.opnfv.org/display/ves/VES+Home) and Models (https://wiki.opnfv.org/display/models/Models+Home). Progress toward ONAP-OPNFV integration will also support the OPNFV Lab As A Service (LaaS) use cases shown at https://wiki.opnfv.org/display/INF/Lab+as+a+Service, which will enable ONAP developers to access OPNFV deployed VIM/NFVI environments (and vice versa). This and other work in the Models project above will help setup a diverse VIM/NFVI environment for supporting the Multi-VIM project in ONAP. Thanks, Bryan Sullivan | AT&T From: opnfv-tech-discuss-boun...@lists.opnfv.org<mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of morgan.richo...@orange.com<mailto:morgan.richo...@orange.com> Sent: Friday, February 09, 2018 8:34 AM To: helen.c...@huawei.com<mailto:helen.c...@huawei.com> Cc: gary.i...@huawei.com<mailto:gary.i...@huawei.com>; ran.pol...@amdocs.com<mailto:ran.pol...@amdocs.com>; feng.xiao...@zte.com.cn<mailto:feng.xiao...@zte.com.cn>; PLATANIA, MARCO (MARCO) <plata...@research.att.com<mailto:plata...@research.att.com>>; onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: [opnfv-tech-discuss] [integration] demo E2E testing & reporting next Tuesday? Hi Helen as discussed 2 weeks ago, I am now ready for a demo (20') to the integration team next Tuesday. The demo deals with: - reuse and adaptation of OPNFV test API to declare/collect projects/pods/testcases/test results (Serena Feng). It is indeed used to collect the results but also to centralize the declaration of the automated testing. - reuse and adaptations of the Functest Framework to integrate robot suites, it should be possible to onboard the vFW and vLB through the VNF abstraction or through the basic bash integration (Cédric Ollivier) One single framework (Functest) to launch functional tests on infra and ONAP tests is a good example of Xcommunity synergies... We could imagine to use additional testing capabilities (Yardstick, bottlenecks,..) that already integrate lots of simulators for benchmarking Yardstick for instance already integrates tests dealing with sample VNFs (https://github.com/opnfv/samplevnf<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_opnfv_samplevnf&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=GlWhzY_Th1uAFmIPcIJ-3BcOVqO9NWOeHbjnHtnwk54&s=6QjGnNd89x3ig38vs5nAzMHgQXqciUaYqIRdj7PPjrI&e=>) - reuse and adaptations of the reporting page, customized for ONAP (Jack & myself) Its provide to ONAP community a way to collect and aggregate the results from different tests distributed on different CI PODs and offer a global status. The demo will be hosted on the Orange OpenLab where we have already 2 ONAP up&running (based on heat Template) and hopefully a third one based on OOM very soon. Some snapshots swagger of the test API <image002.png> view of the reporting page <image003.png> I sliced a little bit the robot suite into 4 different ones (but did not change 1 line of code in the test) I could start sharing the code but integration repo is too wide, we need sub repos (initially in OPNFV we had everything in 1 repo then we had to split...we cans firt get subrepo...) Do you have any update on the LF ticket to get the hosting resources (I hope we coudl setup this env in public env so any lab coudl report results). and for the repos, I think the following ones are needed: - integration/functest - integration/reporting - integration/benchmark /Morgan traces of Robot taces launched from Functest docker container (on the jumphost not from the robot VM) code unchanged, tests sliced into 4 different Robot to distinguish core functions, DCAE, multicloud and the drivers run_tests -t all 2018-02-08 16:40:16,783 - functest.ci.run_tests - INFO - TESTS TO BE EXECUTED: +---------------+---------------+--------------------------+------------------------------------------+-----------------------------------------+ | TIERS | ORDER | CI LOOP | DESCRIPTION | TESTCASES | +---------------+---------------+--------------------------+------------------------------------------+-----------------------------------------+ | smoke | 1 | (daily)|(weekly) | Set of basic Functional tests to | robot_healthcheck robot_dcae | | | | | validate the OPNFV scenarios. | robot_3rdparty robot_multicloud | +---------------+---------------+--------------------------+------------------------------------------+-----------------------------------------+ 2018-02-08 16:40:16,786 - functest.ci.run_tests - INFO - Running tier 'smoke' 2018-02-08 16:40:16,786 - functest.ci.run_tests - INFO - Running test case 'robot_healthcheck'... 2018-02-08 16:40:17,934 - functest.core.robotframework - INFO - ============================================================================== Onap-Healthcheck :: Testing ecomp components are available via calls. ============================================================================== Basic SDNGC Health Check | PASS | ------------------------------------------------------------------------------ Basic A&AI Health Check | PASS | ------------------------------------------------------------------------------ Basic Policy Health Check | PASS | ------------------------------------------------------------------------------ Basic MSO Health Check | PASS | ------------------------------------------------------------------------------ Basic ASDC Health Check | PASS | ------------------------------------------------------------------------------ Basic APPC Health Check | PASS | ------------------------------------------------------------------------------ Basic Portal Health Check | PASS | ------------------------------------------------------------------------------ Basic Message Router Health Check | PASS | ------------------------------------------------------------------------------ Basic VID Health Check | PASS | ------------------------------------------------------------------------------ Basic Microservice Bus Health Check | PASS | ------------------------------------------------------------------------------ Basic CLAMP Health Check | PASS | ------------------------------------------------------------------------------ catalog API Health Check | PASS | ------------------------------------------------------------------------------ nslcm API Health Check | PASS | ------------------------------------------------------------------------------ resmgr API Health Check | PASS | ------------------------------------------------------------------------------ usecaseui-gui API Health Check | PASS | ------------------------------------------------------------------------------ vnflcm API Health Check | PASS | ------------------------------------------------------------------------------ vnfmgr API Health Check | PASS | ------------------------------------------------------------------------------ vnfres API Health Check | PASS | ------------------------------------------------------------------------------ workflow API Health Check | PASS | ------------------------------------------------------------------------------ Onap-Healthcheck :: Testing ecomp components are available via calls. | PASS | 19 critical tests, 19 passed, 0 failed 19 tests total, 19 passed, 0 failed ============================================================================== Output: /home/opnfv/functest/results/robot/output.xml 2018-02-08 16:40:17,934 - functest.core.robotframework - INFO - Results were successfully generated 2018-02-08 16:40:17,963 - functest.core.robotframework - INFO - Results were successfully parsed 2018-02-08 16:40:17,963 - functest.ci.run_tests - INFO - Test result: +---------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +---------------------------+------------------+------------------+----------------+ | robot_healthcheck | functest | 00:01 | PASS | +---------------------------+------------------+------------------+----------------+ 2018-02-08 16:40:17,966 - functest.ci.run_tests - INFO - Running test case 'robot_dcae'... 2018-02-08 16:40:18,738 - functest.core.robotframework - INFO - ============================================================================== Dcae :: Testing ecomp components are available via calls. ============================================================================== Basic DCAE Health Check | FAIL | ConnectionError: HTTPConnectionPool(host='10.4.2.78', port=8080): Max retries exceeded with url: /healthcheck (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f20e07565d0>: Failed to establish a new connection: [Errno 111] Connection refused',)) ------------------------------------------------------------------------------ Dcae :: Testing ecomp components are available via calls. | FAIL | 1 critical test, 0 passed, 1 failed 1 test total, 0 passed, 1 failed ============================================================================== Output: /home/opnfv/functest/results/robot/output.xml 2018-02-08 16:40:18,738 - functest.core.robotframework - INFO - Results were successfully generated 2018-02-08 16:40:18,741 - functest.core.robotframework - INFO - Results were successfully parsed 2018-02-08 16:40:18,741 - functest.ci.run_tests - INFO - Test result: +--------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +--------------------+------------------+------------------+----------------+ | robot_dcae | functest | 00:01 | FAIL | +--------------------+------------------+------------------+----------------+ 2018-02-08 16:40:18,744 - functest.ci.run_tests - ERROR - The test case 'robot_dcae' failed. 2018-02-08 16:40:18,744 - functest.ci.run_tests - INFO - Running test case 'robot_3rdparty'... 2018-02-08 16:40:19,263 - functest.core.robotframework - INFO - ============================================================================== 3Rdparty :: Testing ecomp components are available via calls. ============================================================================== emsdriver API Health Check | PASS | ------------------------------------------------------------------------------ gvnfmdriver API Health Check | PASS | ------------------------------------------------------------------------------ huaweivnfmdriver API Health Check | PASS | ------------------------------------------------------------------------------ nokiavnfmdriver API Health Check | PASS | ------------------------------------------------------------------------------ ztesdncdriver API Health Check | PASS | ------------------------------------------------------------------------------ ztevmanagerdriver API Health Check | FAIL | 502 != 200 ------------------------------------------------------------------------------ 3Rdparty :: Testing ecomp components are available via calls. | FAIL | 6 critical tests, 5 passed, 1 failed 6 tests total, 5 passed, 1 failed ============================================================================== Output: /home/opnfv/functest/results/robot/output.xml 2018-02-08 16:40:19,263 - functest.core.robotframework - INFO - Results were successfully generated 2018-02-08 16:40:19,270 - functest.core.robotframework - INFO - Results were successfully parsed 2018-02-08 16:40:19,271 - functest.ci.run_tests - INFO - Test result: +------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +------------------------+------------------+------------------+----------------+ | robot_3rdparty | functest | 00:00 | FAIL | +------------------------+------------------+------------------+----------------+ 2018-02-08 16:40:19,274 - functest.ci.run_tests - ERROR - The test case 'robot_3rdparty' failed. 2018-02-08 16:40:19,274 - functest.ci.run_tests - INFO - Running test case 'robot_multicloud'... 2018-02-08 16:40:19,533 - functest.core.robotframework - INFO - ============================================================================== Multicloud :: Testing ecomp components are available via calls. ============================================================================== multicloud API Health Check | PASS | ------------------------------------------------------------------------------ multicloud-ocata API Health Check | PASS | ------------------------------------------------------------------------------ multicloud-titanium_cloud API Health Check | PASS | ------------------------------------------------------------------------------ multicloud-vio API Health Check | PASS | ------------------------------------------------------------------------------ Multicloud :: Testing ecomp components are available via calls. | PASS | 4 critical tests, 4 passed, 0 failed 4 tests total, 4 passed, 0 failed ============================================================================== Output: /home/opnfv/functest/results/robot/output.xml 2018-02-08 16:40:19,533 - functest.core.robotframework - INFO - Results were successfully generated 2018-02-08 16:40:19,540 - functest.core.robotframework - INFO - Results were successfully parsed 2018-02-08 16:40:19,540 - functest.ci.run_tests - INFO - Test result: +--------------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +--------------------------+------------------+------------------+----------------+ | robot_multicloud | functest | 00:00 | PASS | +--------------------------+------------------+------------------+----------------+ 2018-02-08 16:40:19,543 - functest.ci.run_tests - INFO - Deployment description: +-------------------------+--------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+--------------------------------------------------------------+ | INSTALLER_TYPE | daisy | | DEPLOY_SCENARIO | os-nosdn-nofeature-ha | | BUILD_TAG | jenkins-functest-kolla-baremetal-daily-amsterdam-666 | | CI_LOOP | daily | +-------------------------+--------------------------------------------------------------+ 2018-02-08 16:40:19,544 - functest.ci.run_tests - INFO - FUNCTEST REPORT: +---------------------------+------------------+---------------+------------------+----------------+ | TEST CASE | PROJECT | TIER | DURATION | RESULT | +---------------------------+------------------+---------------+------------------+----------------+ | robot_healthcheck | functest | smoke | 00:01 | PASS | | robot_dcae | functest | smoke | 00:01 | FAIL | | robot_3rdparty | functest | smoke | 00:00 | FAIL | | robot_multicloud | functest | smoke | 00:00 | PASS | +---------------------------+------------------+---------------+------------------+----------------+ _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer _______________________________________________ onap-discuss mailing list onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org> https://lists.onap.org/mailman/listinfo/onap-discuss IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
