Hi Alec, Please see my answers inline!
Thanks! Gabriel From: Alec Hothan (ahothan) [mailto:ahot...@cisco.com] Sent: Friday, May 05, 2017 11:11 AM To: Yuyang (Gabriel); morgan.richo...@orange.com; Cooper, Trevor; opnfv-tech-discuss@lists.opnfv.org; test...@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] On Stress Test Demo//RE: [test-wg] Notes from OPNFV Plugfest meeting - "Testing group Euphrates collaborative work" Hi Gabriel, Regarding TC1, do you have the results and original chart available somewhere (the chart in the picture and in the slides is kind of small). Gabriel: We do not have the original results and chart now since this is one time result that we have when we do local test. We have discussed the results plotting within Testperf. The test results will be shown in the community ELK maintained by bitergia in Q2. Some results could be found in the community CI, e.g., https://build.opnfv.org/ci/view/bottlenecks/job/bottlenecks-compass-posca_stress_traffic-baremetal-daily-danube/5/console Do you also have a detailed description of the test: Gabriel: I assume you ask detailed about TC3 here, since we only run TC1 on vPOD or bare mental POD. · what kind of packet gen you are running in the first VM and what kind of forwarder you are running in the second VM Gabriel: For TC3, we use netperf to send packet and retransmit it in the second VM. · what flavor Gabriel: Bottlenecks calls yardstick to do each test, the flavor is specified in Yardstick, i.e., yardstick-flavor: nova flavor-create yardstick-flavor 100 512 3 1. · what vswitch was used Gabriel: OVS is used. If you have any recommendation, please also inform us :) · how are the VMs placed (wrt compute node, numa node) Gabriel: VMs are created by using heat client without specifying how and where to place them · I assume the test will ramp up the number of VM pairs, if so how do you synchronize your VM starts/measurements (VMs don’t tend to all start the test at the same time due to the latency for bringing up individual VMs) Gabriel: Yes, the VMs will not start to ping simultaneously. We can only simultaneously sent the request to create all the VMs, waiting for the creation and then to the ping operation. · How do you track drop rate and latency across VM pairs Gabriel: I assume you mean the packet drop rate and latency here. For these metrics, we use netperf to do the monitoring Gabriel: you could also do the test locally. For more information, please refer to the link below. Bottlenecks Testing Guide: http://docs.opnfv.org/en/stable-danube/submodules/bottlenecks/docs/testing/developer/devguide/posca_guide.html Thanks! Alec From: <opnfv-tech-discuss-boun...@lists.opnfv.org<mailto:opnfv-tech-discuss-boun...@lists.opnfv.org>> on behalf of "Yuyang (Gabriel)" <gabriel.yuy...@huawei.com<mailto:gabriel.yuy...@huawei.com>> Date: Thursday, May 4, 2017 at 7:48 PM To: "morgan.richo...@orange.com<mailto:morgan.richo...@orange.com>" <morgan.richo...@orange.com<mailto:morgan.richo...@orange.com>>, "Cooper, Trevor" <trevor.coo...@intel.com<mailto:trevor.coo...@intel.com>>, "opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>" <opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>>, "test...@lists.opnfv.org<mailto:test...@lists.opnfv.org>" <test...@lists.opnfv.org<mailto:test...@lists.opnfv.org>> Subject: [opnfv-tech-discuss] On Stress Test Demo//RE: [test-wg] Notes from OPNFV Plugfest meeting - "Testing group Euphrates collaborative work" Hi, For the stress test demo, we have added more details. Youtube link is provided below. https://youtu.be/TPd4NZr__HI Slides deck is uploaded in the WIKI page: https://wiki.opnfv.org/display/bottlenecks/Sress+Testing+over+OPNFV+Platform The demo contents and results are briefly summarized below. • Testing Contents • Executing Stress Test and Provide comparison results for different installers (Installer A and Installer B) • Up to 100 stacks for Installer A (Completes the test) • Up to 40 stacks for Installer B (System fails to complete the test) • Testing Steps • Enter the Bottlenecks Repo: cd /home/opnfv/bottlenecks • Prepare virtual environment: . pre_virt_env.sh • Executing ping test case: bottlenecks testcase run posca_factor_ping • Clean the virtual environment: . rm_virt_env.sh • Testing Results • Testing for Installer A • Up to 100 stacks in configuration file for Installer B • 1 stack SSH error when number of stack raised to 50 • When stack number up to 100, most of the errors are heat response time out • 100 stacks are established successfully in the end • Testing for Installer B • Up to 40 stacks in configuration file for Installer B • When stack number up to 30, the system fails to create all the stacks • 21 stacks are either created failure or keeping in creation • To verify the system performance, we choose to do clean-up and run the test again • When stack number up to 20, same situation happens as in the last test • The system performance degrades • Different to the test for Installer A, we do the verification step because the system clearly malfunctions. • Which not shown in the demo is that after 3 rounds of the stress test, the system fail even to create 5 stacks Best, Gabriel From: test-wg-boun...@lists.opnfv.org<mailto:test-wg-boun...@lists.opnfv.org> [mailto:test-wg-boun...@lists.opnfv.org] On Behalf Of morgan.richo...@orange.com<mailto:morgan.richo...@orange.com> Sent: Tuesday, May 02, 2017 3:47 PM To: Cooper, Trevor; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>; test...@lists.opnfv.org<mailto:test...@lists.opnfv.org> Subject: Re: [test-wg] [opnfv-tech-discuss] Notes from OPNFV Plugfest meeting - "Testing group Euphrates collaborative work" Thnaks Trevor I added some points that reflect the notes for the next test weekly meeting planned on the 4th of May: https://wiki.opnfv.org/display/meetings/TestPerf feel free to add additional points I will not be able to join this time, could you chair the meeting? /Morgan Le 28/04/2017 à 16:36, Cooper, Trevor a écrit : Status of Danube and what improved 1. landing page - http://testresults.opnfv.org/reporting2/reporting/index.html - http://testresults.opnfv.org/reporting2/reporting/index.html#!/landingpage/table § Meaning of info displayed? Test ran, test passed … agree for consistency … TBD 2. catalogue - diagram with roll-over - http://testresults.opnfv.org/reporting2/reporting/index.html#!/select/visual - All PTLs to review if test cases are valid and if not remove - Add short description that is human readable (add API field) ○ Define test domain categories - start by using labels on test ecosystem diagram 3. Stress tests - video of presentation is available - simultaneous vPING which increases until system fails 4. Documentation ○ Ecosystem diagram ○ Testing guide ○ Add agenda item to TWG in June (2 sessions on how to improve) ○ New test projects … add to common docs ○ Developer guide - move to docs 5. Reporting status / dashboard ○ Bottlenecks TBD? 6. Bitergia ○ Morgan meeting 10th May ○ https://wiki.opnfv.org/display/testing/Result+alignment+for+ELK+post-processing ○ Add to TWG agenda next week to revisit Wish list for Infra group - One POD dedicated to long duration tests (with reservation mechanism) - Per installer ○ Stable of previous release ○ Master - OPNFV POD on demand for tester (before merging) ○ Per installer if possible ○ Infra group working on it ○ Today can fill a ticket to ask if there is a free resource Micro services - Deploy VNF ○ Retrieve image ○ Deploy ○ Prebuild image with tools - standardise on a framework ○ Take image and copy over tools - Ansible (infra doing this with Open Stack Ansible) ○ Catalog of roles for Ansible? § Ansible Galaxy is a tool for deploying and managing roles □ e.g. Install TREX TG □ Turn on live migration in Nova - Test generator? - Collect / display results ○ Test API and DB ○ Reuse collectd lib for VNF KPIs - Analytics of results - What APIs to expose to other test projects? ○ Functest § Deploy orchestrator VNF § Use Traffic gen and generate load /Trevor _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss -- Morgan Richomme Orange/ IMT/ OLN/ CNC/ NCA/ SINA Network architect for innovative services Future of the Network community member Open source Orange community manager tel. +33 (0) 296 072 106 mob. +33 (0) 637 753 326 morgan.richo...@orange.com<mailto:morgan.richo...@orange.com> _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
